Posted inAI AWS System Design

Multi-Tenant Coding Assistants: Isolated ECS Fargate Spot Runtimes

1 Comment
Multi-Tenant Coding Assistants: Isolated ECS Fargate Spot Runtimes
8 views

This guide covers multi-tenant coding assistants with production-focused examples and trade-offs.

Shared agent fleets fail multi-tenant SaaS the boring way: one customer’s overnight refactor storm saturates CPU, fills /tmp, and exhausts Bedrock quotas for everyone. Isolate runtimes per tenant on Fargate Spot with their own IAM role, disk quota, and concurrency caps.
⚡ TL;DR: One task definition family, many tenant-scoped tasks: task role via STS/AssumeRole with tenant tag, ephemeral storage caps, Security Group egress allowlists, Spot with on-demand fallback for paying tiers. Pair with Secure AI Sandboxes and Bedrock Retrieval Filters for tenants.

Isolation tiers that actually ship — multi-tenant coding assistants

Tier Runtime When
Pool Shared warm workers, tenant tag on session Free / trial
Soft silo Per-tenant Fargate Spot tasks Growth
Hard silo Dedicated account / VPC cell Enterprise
Most products need soft silo first. Pool-only designs leak through /tmp, Bedrock TPM, and outbound git clones even when IAM looks fine.

Per-tenant task role

json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "arn:aws:s3:::acme-tenant-${aws:PrincipalTag/tenant_id}/*",
    "Condition": {
      "StringEquals": { "aws:PrincipalTag/tenant_id": "${aws:PrincipalTag/tenant_id}" }
    }
  }]
}
typescript
// orchestrator
const creds = await sts.assumeRole({
  RoleArn: TENANT_RUNTIME_ROLE,
  RoleSessionName: `agent-${tenantId}-${sessionId}`,
  Tags: [{ Key: "tenant_id", Value: tenantId }],
  DurationSeconds: 3600,
});
await ecs.runTask({
  capacityProviderStrategy: [
    { capacityProvider: "FARGATE_SPOT", weight: 4 },
    { capacityProvider: "FARGATE", weight: 1, base: 0 },
  ],
  overrides: {
    cpu: "1024",
    memory: "2048",
    ephemeralStorage: { sizeInGiB: 21 },
  },
  tags: [
    { key: "tenant_id", value: tenantId },
    { key: "session_id", value: sessionId },
  ],
});
✅ Tenant tag on session; S3 paths scoped.
❌ Shared task role with Resource: * and hope. Bootstrap the agent with short-lived credentials from the orchestrator over the task ENI — do not rely on a long-lived shared task role that can reach every tenant bucket.

Noisy-neighbor controls

Control Setting
Concurrent tasks / tenant Soft 5 / hard 20
Bedrock TPM / tenant Inference profile + usage plan
Ephemeral disk 21–40 GiB + tmp cleaner
Egress VPC endpoints + domain allowlist
Wall clock Stop task at 30–60 min
Clone size Reject repos over N GiB before clone
Emit per-tenant CloudWatch metrics (RunningTaskCount, BedrockInvocations, EphemeralBytes). Alarm when one tenant consumes >N% of fleet. Cost attribution mirrors cross-account Bedrock patterns.
typescript
// admission control
async function admit(tenantId: string) {
  const running = await countTasks(tenantId);
  if (running >= softLimit(tenantId)) await enqueue(tenantId);
  if (running >= hardLimit(tenantId)) throw new Error("tenant_concurrency_exhausted");
}

Spot interruption handling

typescript
// listen for Spot interruption notice via container metadata / EventBridge
async function onSpotInterrupt(taskArn: string) {
  await checkpointSession(taskArn); // prompts + tool log to S3
  await rescheduleOnDemand(taskArn); // paid tier: Fargate base
}
Checkpoint format should be replayable: messages, tool results, git SHA, and env digests. Deterministic replay of interrupted sessions: Deterministic Replay.

Network and secret boundaries

  • Private subnets only; no public IP on agent tasks
  • VPC endpoints for S3, ECR, Bedrock, STS, Logs
  • Domain allowlist proxy for github.com / package registries — deny everything else
  • Disable IMDS for the app container (httpHopLimit / task role only) so a prompt-injected agent cannot steal credentials via 169.254.169.254
Secrets for package tokens: inject via SSM/Secrets Manager at task start scoped by tenant tag, never bake into the image.

Observability and abuse signals

Watch for:
  • Sudden clone of huge monorepos (egress + disk)
  • Tool loops calling the same shell command hundreds of times
  • Cross-tenant S3 AccessDenied spikes (confused deputy attempts)
Feed those into the same ChatOps draft flow from Incident ChatOps AI — draft tickets, don’t auto-kill without policy.

Cost and quota attribution

Tag every Bedrock invoke, ECS task, and S3 put with tenant_id. Export CUR/Cost Explorer splits weekly. When a tenant approaches their included agent-hours, degrade to smaller models or queue rather than stealing Spot capacity from neighbors. Publish a tenant-facing usage meter so finance conversations are not forensic archaeology.
typescript
bedrock.invoke({
  modelId,
  // ...
  // request metadata / logging fields
  trace: { tenantId, sessionId, surface: "coding-agent" },
});
Pair quotas with LLM Cost Controls so per-engineer budgets still apply inside a tenant.

Closing checklist

  • [ ] Per-tenant IAM via session tags; no shared writable buckets
  • [ ] Fargate Spot + on-demand fallback by tier
  • [ ] Concurrency and Bedrock budgets per tenant
  • [ ] Egress allowlist; no IMDS from app container (hop limit / task ENI)
  • [ ] Checkpoint on Spot interrupt
  • [ ] Cell/bulkhead story documented for enterprise tenants
  • [ ] Admission control soft/hard limits enforced in orchestrator
  • [ ] Ephemeral disk + tmp cleaner; reject oversized clones

Related reading

Last updated on September 11, 2026

Related: Bedrock Model Distillation: Shrink Coding Assistants for Edge Latency · Day 70: Project: Secure Coding-Agent Sandbox on ECS Fargate · CodeBuild Spec Sandboxes for AI Coding Agents: Ephemeral Builds Without ECS Sprawl

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Most viewed

  1. Python Decorators Explained: From Simple Wrappers to Production Patterns164 views
  2. Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three80 views
  3. AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API61 views
  4. Python String Methods: Every str Method With Real Production Examples53 views
  5. Css Animation Performance: Performance-Optimized Motion for Modern UIs44 views

Newly added

  1. AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
  2. Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
  3. Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
  4. AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
  5. Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.

Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

1 Comment

Leave a comment

No account needed. Name and email are optional.