21 views
You have a backlog of open pull requests and a repository scan that should be finished before standup. Running the agent from a laptop means someone babysits it. A single Lambda function stops at 15 minutes, which is not enough for a deep review that clones a large repository and runs tests. Starting EC2 instances by hand works until someone forgets to stop one.
AWS Batch is built for this kind of work. You describe the job once (a container image, CPU, memory, roles), put work on a queue, and Batch provisions compute, runs the jobs, retries the ones that fail for infrastructure reasons, and scales back down when the queue is empty. This guide covers two shapes of coding-agent work on Batch: wide fleets (one child job per pull request or repository, using array jobs) and long single jobs (a migration that runs for hours). It replaces our earlier post on long-running Batch jobs, so both patterns now live here.
TL;DR
- Use a Fargate or Fargate Spot compute environment unless you need GPUs, a custom AMI, more than 32 vCPUs or more than 244 GiB of memory per job.
- Submit one array job (size 2 to 10,000) instead of thousands of single jobs. Each child reads
AWS_BATCH_JOB_ARRAY_INDEXto pick its pull request.- Always set a timeout. Batch has no default timeout, and a job that times out is not retried.
- Write a retry strategy that exits on permanent errors. If no
evaluateOnExitrule matches, Batch retries.- Make every child idempotent, so a retried attempt never posts the same review comment twice.
The building blocks, in plain terms
Batch has four concepts. If you know these, the rest of the service makes sense.- Job definition: the template for a job: container image, vCPU and memory, IAM roles, environment variables, secrets, retry strategy and timeout. Think of it as a versioned recipe.
- Job queue: where submitted jobs wait. A queue has a priority and an ordered list of up to three compute environments.
- Compute environment: the capacity jobs run on. Managed environments can be EC2, Spot, Fargate or Fargate Spot. A single queue can’t mix EC2-based and Fargate-based environments.
- Job: one submitted unit of work. An array job is a parent that fans out into child jobs which share the same definition but receive different index numbers.
Wide or long: pick the shape first
| Shape | Example | Batch feature | Watch out for |
|---|---|---|---|
| Wide (many small units) | Review every open PR, scan 400 repositories | Array job with an S3 manifest | Manifest length must match the array size |
| Long (one big unit) | Migrate a monorepo to a new framework | Single job with a long timeout and checkpoints | Spot interruptions; Fargate jobs shouldn’t expect to run more than 14 days |
| Ordered pipeline | Scan, then open issues for findings | Job dependencies (N_TO_N, SEQUENTIAL) or Step Functions | A job can have at most 20 dependencies |
| Interactive | Chat or IDE requests | Not Batch; use Lambda or a service | Queueing adds seconds to minutes of start latency |
Before you start
- A container image for your agent in Amazon ECR, with git, your language toolchains and an entrypoint that exits with meaningful codes.
- A VPC with private subnets. Fargate jobs in subnets without a NAT gateway need
assignPublicIpset toENABLED, or VPC endpoints for the services they call (ECR, S3, CloudWatch Logs, Secrets Manager, Bedrock). - Two IAM roles: an execution role that lets Batch pull the image, write logs and fetch secrets, and a job role that the agent code uses (for example,
bedrock:InvokeModeland access to its S3 prefix).
Step 1: Create compute environments and a queue
Create a Fargate Spot environment for cheap, interruptible capacity and an on-demand Fargate environment as a fallback. The queue tries environments in ascending order, so Spot is used first.bash
aws batch create-compute-environment \
--compute-environment-name agents-fargate-spot \
--type MANAGED \
--state ENABLED \
--compute-resources type=FARGATE_SPOT,maxvCpus=64,subnets=subnet-aaa,subnet-bbb,securityGroupIds=sg-agents
aws batch create-compute-environment \
--compute-environment-name agents-fargate \
--type MANAGED \
--state ENABLED \
--compute-resources type=FARGATE,maxvCpus=32,subnets=subnet-aaa,subnet-bbb,securityGroupIds=sg-agents
aws batch create-job-queue \
--job-queue-name agents-overnight \
--state ENABLED \
--priority 10 \
--compute-environment-order order=1,computeEnvironment=agents-fargate-spot order=2,computeEnvironment=agents-fargatemaxvCpus is your hard concurrency ceiling. With 2 vCPUs per child, a 64 vCPU Spot environment runs at most 32 children at once. That number also bounds how fast you call Bedrock, so size it with your model quotas in mind. Fargate environments don’t accept EC2-only settings such as instanceTypes, allocationStrategy, minvCpus or bidPercentage. If you move to EC2 Spot later, use type=SPOT with the SPOT_PRICE_CAPACITY_OPTIMIZED allocation strategy, which the docs recommend over SPOT_CAPACITY_OPTIMIZED in most cases.
Step 2: Register the job definition
json
Register it with {
"jobDefinitionName": "agent-pr-review",
"type": "container",
"platformCapabilities": ["FARGATE"],
"containerProperties": {
"image": "111122223333.dkr.ecr.ap-south-1.amazonaws.com/coding-agent:1.8.0",
"command": ["python", "-m", "agent.review"],
"resourceRequirements": [
{"type": "VCPU", "value": "2"},
{"type": "MEMORY", "value": "4096"}
],
"executionRoleArn": "arn:aws:iam::111122223333:role/agents-batch-execution",
"jobRoleArn": "arn:aws:iam::111122223333:role/agents-batch-job",
"networkConfiguration": {"assignPublicIp": "DISABLED"},
"fargatePlatformConfiguration": {"platformVersion": "LATEST"},
"environment": [
{"name": "PROMPT_VERSION", "value": "review-v3"}
],
"secrets": [
{"name": "GITHUB_TOKEN", "valueFrom": "arn:aws:secretsmanager:ap-south-1:111122223333:secret:agents/github"}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/aws/batch/agents",
"awslogs-region": "ap-south-1",
"awslogs-stream-prefix": "pr-review"
}
}
},
"retryStrategy": {
"attempts": 3,
"evaluateOnExit": [
{"onExitCode": "3", "action": "EXIT"},
{"onExitCode": "4", "action": "EXIT"}
]
},
"timeout": {"attemptDurationSeconds": 1800}
}aws batch register-job-definition --cli-input-json file://agent-pr-review.json. Three details are worth understanding:
- Retry rules.
attemptscan be 1 to 10. WhenevaluateOnExitis set and no rule matches, Batch retries. Here the agent exits with code 3 for “bad input, don’t retry” and 4 for “policy refused”, and everything else (a Spot interruption, a crashed container, an internal error) gets another attempt. If you prefer opt-in retries, end the list with{"onReason": "*", "action": "EXIT"}as the Batch user guide shows. - Timeouts. Batch has no default timeout. The minimum is 60 seconds. When the time is up, the container gets
SIGTERM, thenSIGKILL30 seconds later. A job terminated by its timeout is not retried, so a stuck tool loop fails fast rather than burning three attempts. - Secrets. The execution role needs
secretsmanager:GetSecretValueon the referenced secret. The token arrives as an environment variable and never appears in the job definition.
Step 3: Submit an array job from a manifest
An array job has a size between 2 and 10,000. Batch gives each child the same job ID with its index appended (for examplejobid:42) and sets AWS_BATCH_JOB_ARRAY_INDEX, starting at 0. The submitter writes the list of work to S3 first, then submits once. One SubmitJob call creates the whole fleet, which matters because SubmitJob is limited to 50 transactions per second per account.
python
Note the import datetime
import json
import boto3
s3 = boto3.client("s3")
batch = boto3.client("batch")
BUCKET = "agents-overnight"
MAX_ARRAY = 10_000
def submit_overnight(prs: list[dict]) -> list[str]:
"""Write one manifest per chunk and submit one array job per manifest."""
day = datetime.date.today().isoformat()
job_ids = []
for start in range(0, len(prs), MAX_ARRAY):
chunk = prs[start:start + MAX_ARRAY]
key = f"manifests/{day}/prs-{start:05d}.json"
s3.put_object(Bucket=BUCKET, Key=key, Body=json.dumps({"prs": chunk}).encode())
request = {
"jobName": f"pr-review-{day}-{start:05d}",
"jobQueue": "agents-overnight",
"jobDefinition": "agent-pr-review",
"containerOverrides": {
"environment": [{"name": "MANIFEST_KEY", "value": key}]
},
"tags": {"workload": "coding-agent", "batch": "overnight"},
"propagateTags": True,
}
if len(chunk) >= 2:
request["arrayProperties"] = {"size": len(chunk)}
job_ids.append(batch.submit_job(**request)["jobId"])
return job_ids
def handler(event, context):
prs = list_open_prs() # your forge API call, returns [{"repo": ..., "number": ..., "sha": ...}]
if not prs:
return {"submitted": []}
return {"submitted": submit_overnight(prs)}len(chunk) >= 2 check: an array of size 1 is invalid, so a single pull request becomes a normal job.
Step 4: Write an idempotent worker
Each child runs the same container. It finds its item from the index, checks whether this exact review already exists, does the work, and records completion. Because a retry can happen after a comment was posted but before the job finished, the idempotency key is written before posting, with a conditional write.python
import json
import os
import signal
import sys
import boto3
from botocore.exceptions import ClientError
s3 = boto3.client("s3")
ledger = boto3.resource("dynamodb").Table("agent-review-ledger")
PERMANENT_FAILURE = 3 # matches the EXIT rule in the job definition
stopping = False
def on_sigterm(signum, frame):
# Timeout or Spot interruption: finish the current step, checkpoint, exit.
global stopping
stopping = True
signal.signal(signal.SIGTERM, on_sigterm)
def main() -> int:
index = int(os.environ.get("AWS_BATCH_JOB_ARRAY_INDEX", "0"))
attempt = int(os.environ.get("AWS_BATCH_JOB_ATTEMPT", "1"))
body = s3.get_object(Bucket="agents-overnight", Key=os.environ["MANIFEST_KEY"])["Body"].read()
prs = json.loads(body)["prs"]
if index >= len(prs):
print(f"index {index} outside manifest of {len(prs)}")
return PERMANENT_FAILURE
pr = prs[index]
key = f"{pr['repo']}#{pr['number']}#{pr['sha']}#{os.environ['PROMPT_VERSION']}"
try:
ledger.put_item(
Item={"review_key": key, "status": "claimed", "attempt": attempt},
ConditionExpression="attribute_not_exists(review_key) OR #s <> :done",
ExpressionAttributeNames={"#s": "status"},
ExpressionAttributeValues={":done": "posted"},
)
except ClientError as err:
if err.response["Error"]["Code"] == "ConditionalCheckFailedException":
print(f"{key} already posted; skipping")
return 0
raise
review = run_agent_review(pr, should_stop=lambda: stopping) # your agent loop
if review is None:
return 1 # interrupted: non-zero, not 3, so Batch retries
post_review_comment(pr, review, external_id=key) # forge call keyed by external_id
ledger.update_item(
Key={"review_key": key},
UpdateExpression="SET #s = :done",
ExpressionAttributeNames={"#s": "status"},
ExpressionAttributeValues={":done": "posted"},
)
return 0
if __name__ == "__main__":
sys.exit(main())Step 5: Schedule it in your time zone
EventBridge Scheduler is the clock and Batch is the executor. Schedules accept a time zone, so “2 AM in Bengaluru” stays 2 AM without UTC arithmetic. The schedule invokes the submitter Lambda from Step 3.bash
aws scheduler create-schedule \
--name agents-overnight-pr-review \
--schedule-expression "cron(0 2 * * ? *)" \
--schedule-expression-timezone "Asia/Kolkata" \
--flexible-time-window '{"Mode": "FLEXIBLE", "MaximumWindowInMinutes": 15}' \
--target '{"Arn": "arn:aws:lambda:ap-south-1:111122223333:function:agents-submitter", "RoleArn": "arn:aws:iam::111122223333:role/scheduler-invoke-submitter"}'Long single jobs: migrations that run for hours
Some agent work doesn’t split. A framework migration might edit hundreds of files and then run the full test suite. Lambda’s 15-minute limit and Step Functions Express’s 5-minute limit rule those out, and chaining Lambda functions that re-invoke themselves is fragile and hard to observe. Run it as one Batch job with a timeout that fits the work (for example"attemptDurationSeconds": 28800 for eight hours) and design it to resume:
- Push work to a branch at milestones, and store a cursor (last processed package or file) in DynamoDB or S3.
- On start, read
AWS_BATCH_JOB_ATTEMPT. If it’s greater than 1, resume from the cursor instead of starting over. - Handle
SIGTERMby finishing the current file, saving the cursor and exiting non-zero so the retry rule applies. Remember that a timeout kill is not retried, so set the timeout generously and let your own soft deadline fire first.
.sync integration. The state machine waits for the Batch job to finish and fails the state if the job fails:
json
{
"Migrate": {
"Type": "Task",
"Resource": "arn:aws:states:::batch:submitJob.sync",
"Arguments": {
"JobName": "agent-migrate-monorepo",
"JobQueue": "arn:aws:batch:ap-south-1:111122223333:job-queue/agents-overnight",
"JobDefinition": "arn:aws:batch:ap-south-1:111122223333:job-definition/agent-long-runner:4",
"Timeout": {"AttemptDurationSeconds": 28800}
},
"Next": "OpenPullRequest"
}
}Worked example: 180 pull requests overnight
Here’s how one night plays out with the setup above (the numbers are illustrative):- At 02:00 IST the schedule invokes the submitter. It finds 180 open pull requests, writes
manifests/2026-10-06/prs-00000.jsonand submits one array job of size 180. - Each child asks for 2 vCPUs, so the 64 vCPU Spot environment runs up to 32 children at a time. The queue spills to on-demand Fargate only when Spot capacity can’t be placed.
- Child 17 loses its Spot capacity mid-review. No retry rule matches, so Batch moves it back to
RUNNABLEfor attempt 2. The ledger shows the review was claimed but never posted, so the new attempt runs the review again and posts exactly once. - Child 63 belongs to a pull request whose branch was deleted. The worker exits with code 3, which hits the
EXITrule, so there is no wasted retry. - Child 99 gets stuck in a tool loop. At 30 minutes Batch sends
SIGTERM, the worker stops, and the attempt fails without a retry. - Because two children failed, the parent array job ends as
FAILEDonce every child reaches a terminal state. An EventBridge rule on Batch job state change events sends the failed child IDs to your team channel, and the morning shift reruns just those two pull requests.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Jobs sit in RUNNABLE | No capacity can be placed: maxvCpus reached, the job asks for more than the environment allows, or the environment is invalid | Read statusReason in DescribeJobs; consider jobStateTimeLimitActions on the queue to cancel stuck jobs |
| Container never starts (image pull fails) | Private subnet without NAT or VPC endpoints, and assignPublicIp is DISABLED | Add a NAT gateway or ECR, S3 and Logs endpoints |
SubmitJob rejects the array size | Size is 1 or above 10,000 | Submit single items as normal jobs; chunk large manifests |
| Same bad PR retried three times | No EXIT rule for permanent errors | Exit with a dedicated code and match it with onExitCode |
| A stuck job was never retried | It was terminated by its timeout, which Batch doesn’t retry | Add an in-app soft deadline that exits non-zero before the hard timeout |
| Throttling errors from the submitter | Many single SubmitJob calls (50 TPS limit) | Use one array job per manifest |
| Secret not found at start | Execution role lacks secretsmanager:GetSecretValue | Grant it on the secret ARN to the execution role, not the job role |
| Can’t add a compute environment to the queue | Mixing EC2 and Fargate environments, or more than three | Use separate queues per capacity type |
Production checklist
- ☐ Fargate or Fargate Spot compute environments with a deliberate
maxvCpus - ☐ Job definition pins an image tag or digest; secrets come from Secrets Manager through the execution role
- ☐ Timeout set on every job; in-app soft deadline fires first
- ☐ Retry strategy with explicit
EXITrules for permanent errors - ☐ Array jobs fed by an S3 manifest; single items submitted as normal jobs
- ☐ Idempotency ledger so retries never duplicate comments or pull requests
- ☐ Job role scoped to the agent’s S3 prefix and the Bedrock models it needs
- ☐ Tags (
workload,batch) propagated for cost reports, plus an AWS Budgets alert - ☐ EventBridge rule on job state changes for failure alerts
FAQ
Fargate or EC2 compute for coding agents?
Fargate for most agents: less to patch, per-job isolation and faster starts. Choose EC2 for GPUs, custom AMIs, jobs above 32 vCPUs or 244 GiB, or very high concurrency.Why not Step Functions Map with Lambda?
It works well when every item finishes within Lambda’s 15 minutes and needs no heavy toolchain. Large checkouts, test suites and multi-hour work fit Batch better. You can combine them: Step Functions for the workflow, Batch for the heavy steps.How is Batch different from CodeBuild for this?
CodeBuild is shaped around builds and buildspecs. Batch is a general job scheduler with queues, priorities, array jobs and retry rules, which suits fan-out fleets better.What happens if I cancel the parent array job?
All child jobs are cancelled or terminated with it. You can also cancel individual children without affecting the rest, but the parent will then end asFAILED.
Does Spot make sense for agent work?
Yes, for work that can be retried safely. Idempotent children and checkpointed long jobs turn an interruption into a retry, not lost work. Keep deadline-critical jobs on on-demand capacity.Official documentation
- What is AWS Batch?
- Array jobs (size limits, index variable, parent status)
- Automated job retries and EvaluateOnExit
- Job timeouts
- When to use Fargate and Compute environments on Fargate
- AWS Batch service quotas
- Jobs stuck in RUNNABLE
- Run AWS Batch workloads with Step Functions
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- Python String Methods: Every str Method With Real Production Examples
- API Debugging in 30 Seconds: The Fastest Ways to Fix API Issues
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.