19 views
You need a stable HTTPS endpoint for “create coding-agent session,” tool webhooks, and a thin BFF in front of Bedrock — not another weekend wrestling ALB sticky sessions. AWS App Runner is the unfair advantage for those control-plane APIs: point it at an ECR image (or GitHub repo), get a managed URL, concurrency-based autoscaling, and rolling deploys without owning clusters. Keep Fargate Spot for ephemeral sandboxes and CodeBuild for heavy builds; use App Runner for the always-on HTTP edge. Lock down egress with VPC connectors + Network Firewall patterns, and rate-limit public routes with API Gateway + WAF when you need edge policy App Runner does not provide.
⚡ TL;DR: Deploy agent control-plane APIs on App Runner from ECR; set CPU/mem, concurrency autoscaling, and health checks. Attach instance role for Bedrock/S3/Secrets — no long-lived keys. Use VPC connector for private RDS/Redis. Put WAF/API Gateway in front for public internet abuse cases. Related: Fargate Spot sandboxes, WAF, PrivateLink Bedrock, Budgets.
Where App Runner fits in an agent platform
| Workload | Prefer | Why |
|---|---|---|
| Session API / BFF / webhooks | App Runner | Managed HTTPS + scale-to-low |
| Multi-minute sandbox builds | Fargate Spot / CodeBuild | Cheap ephemeral CPU |
| Fan-out async tool graphs | Step Functions + Lambda | Orchestration |
| Public abuse-prone APIs | API Gateway + WAF → App Runner | Edge policy |
Deploy from ECR with an instance role
bash
# ✅ create App Runner service from ECR image
aws apprunner create-service --cli-input-json '{
"ServiceName": "coding-agent-api",
"SourceConfiguration": {
"AuthenticationConfiguration": {
"AccessRoleArn": "arn:aws:iam::111122223333:role/AppRunnerECRAccessRole"
},
"ImageRepository": {
"ImageIdentifier": "111122223333.dkr.ecr.us-east-1.amazonaws.com/coding-agent-api:1.4.2",
"ImageRepositoryType": "ECR",
"ImageConfiguration": {
"Port": "8080",
"RuntimeEnvironmentVariables": {
"ENV": "prod",
"BEDROCK_REGION": "us-east-1"
}
}
},
"AutoDeploymentsEnabled": false
},
"InstanceConfiguration": {
"Cpu": "1 vCPU",
"Memory": "2 GB",
"InstanceRoleArn": "arn:aws:iam::111122223333:role/CodingAgentApiInstanceRole"
},
"HealthCheckConfiguration": {
"Protocol": "HTTP",
"Path": "/healthz",
"Interval": 10,
"Timeout": 5,
"HealthyThreshold": 1,
"UnhealthyThreshold": 5
}
}'
typescript
// ✅ Express health + Bedrock invoke via instance role (no static keys)
import express from "express";
import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
const app = express();
app.use(express.json({ limit: "1mb" }));
const bedrock = new BedrockRuntimeClient({});
app.get("/healthz", (_req, res) => res.status(200).send("ok"));
app.post("/v1/sessions", async (req, res) => {
// validate JWT / API key first — never trust raw internet body
const tenantId = req.header("x-tenant-id");
if (!tenantId) return res.status(401).json({ error: "unauthorized" });
// enqueue sandbox on Fargate Spot; do not exec user code here
res.status(201).json({ sessionId: crypto.randomUUID(), tenantId });
});
app.post("/v1/complete", async (req, res) => {
const out = await bedrock.send(new InvokeModelCommand({
modelId: "anthropic.claude-3-5-sonnet-20241022-v2:0",
contentType: "application/json",
accept: "application/json",
body: Buffer.from(JSON.stringify(req.body.payload)),
}));
res.json(JSON.parse(new TextDecoder().decode(out.body)));
});
app.listen(8080);
Autoscaling on concurrency (not guesswork)
App Runner scales on concurrent requests per instance. For agent APIs with long Bedrock calls, lower max concurrency per instance so one slow invoke does not head-of-line block everything.bash
aws apprunner update-service --service-arn "$ARN" --auto-scaling-configuration-arn "$AS_ARN"
# Create AutoScalingConfiguration with MaxConcurrency=25, MaxSize=20, MinSize=1
| Traffic shape | MaxConcurrency hint | Notes |
|---|---|---|
| Fast JSON BFF | 50–100 | Short handlers |
| Bedrock proxy | 10–25 | Long TTFT |
| Webhook burst | 50 + MinSize≥1 | Avoid cold start storm |
VPC connector for private dependencies
When the API needs ElastiCache, RDS Proxy, or PrivateLink Bedrock endpoints:bash
Route egress through the same patterns as sandboxes (Network Firewall). Prefer PrivateLink to Bedrock so model calls never hairpin the public internet.
# ✅ VPC connector — place in private subnets with egress control
aws apprunner create-vpc-connector \
--vpc-connector-name coding-agent-api-vpc \
--subnets subnet-aaa subnet-bbb \
--security-groups sg-agent-api
App Runner vs ECS/Fargate vs Lambda for agent APIs
| Concern | App Runner | ECS/Fargate | Lambda |
|---|---|---|---|
| Ops overhead | Lowest | Medium | Lowest (cold starts) |
| Long Bedrock SSE | Good | Good | Awkward timeouts |
| Custom networking | VPC connector | Full | VPC config |
| Untrusted code | ❌ Don’t | Spot tasks ✅ | ❌ Don’t |
| Min cost at idle | Low (can scale in) | Task tax | Per-request |
Hardening checklist for public agent APIs
- [ ] AuthN on every route (JWT / mTLS / API keys in Secrets Manager)
- [ ] Instance role least privilege; no
*on S3/Bedrock - [ ] Request size limits; timeouts; idempotency keys on creates
- [ ] WAF or API Gateway in front if exposed beyond trusted clients
- [ ] Structured logs with
tenant_id; no secrets in stdout - [ ] Health check does not touch Bedrock (avoid flapping on model outages)
- [ ] Auto deploy from
mainonly after CI image scan (Inspector) - [ ] Kill switch via AppConfig for tool routes
python
# ✅ deny-by-default IAM fragment for instance role
# Allow bedrock:InvokeModel on specific model ARNs only
# Allow s3:PutObject on tenants/${aws:PrincipalTag/tenant_id}/*
# Deny s3:DeleteObject, iam:*, organizations:*
Production checklist
- [ ] ECR image pinned by digest in prod deploys
- [ ] Instance role + ECR access role separated
- [ ] Health
/healthzcheap and dependency-light - [ ] Autoscaling config tuned for Bedrock latency
- [ ] VPC connector for private data stores
- [ ] Budgets alarm on App Runner + Bedrock
- [ ] Sandboxes remain on Fargate Spot / CodeBuild — not in-process
- [ ] Security Hub / Inspector on images; findings ticketed
FAQ
Q: Can App Runner replace all ECS?A: No — keep ECS/Fargate for privileged networking, GPU, and untrusted sandboxes. App Runner shines for managed HTTP services. Q: Source code repo vs ECR?
A: ECR + your CI gives reproducible digests and scanning. Repo-based builds are fine for prototypes; pin images for prod agent APIs. Q: WebSockets?
A: App Runner is request/response HTTP oriented. For multi-turn streaming UIs, prefer API Gateway WebSockets or ALB — see prior CheatCoders WebSocket agent patterns.
Observability for agent API services
Emit EMF or OpenTelemetry spans withtenant_id, route, and model_id. Alarm on p95 latency and 5xx — not on Bedrock token count alone (that belongs in cost dashboards). Trace multi-hop tool calls with ADOT patterns from prior CheatCoders posts so App Runner is one service in the graph, not a black box.
Related reading
- AWS Fargate Spot: Cheap Ephemeral Sandboxes
- API Gateway + WAF Rate Limits
- PrivateLink Bedrock
- AWS Budgets + Cost Anomaly
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- Python String Methods: Every str Method With Real Production Examples
- API Debugging in 30 Seconds: The Fastest Ways to Fix API Issues
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.