Multi-Tenant Coding Assistants: Isolated ECS Fargate Spot Runtimes

Shared agent fleets fail multi-tenant SaaS the boring way: one customer’s overnight refactor storm saturates CPU, fills /tmp, and exhausts Bedrock quotas for everyone. Isolate runtimes per tenant on Fargate Spot with their own IAM role, disk quota, and concurrency caps.

⚡ TL;DR: One task definition family, many tenant-scoped tasks: task role via STS/AssumeRole with tenant tag, ephemeral storage caps, Security Group egress allowlists, Spot with on-demand fallback for paying tiers. Pair with Secure AI Sandboxes and Bedrock Retrieval Filters for tenants.

Isolation tiers that actually ship

Most products need soft silo first. Pool-only designs leak through /tmp, Bedrock TPM, and outbound git clones even when IAM looks fine.

Per-tenant task role

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "arn:aws:s3:::acme-tenant-${aws:PrincipalTag/tenant_id}/*",
    "Condition": {
      "StringEquals": { "aws:PrincipalTag/tenant_id": "${aws:PrincipalTag/tenant_id}" }
    }
  }]
}

// orchestrator
const creds = await sts.assumeRole({
  RoleArn: TENANT_RUNTIME_ROLE,
  RoleSessionName: `agent-${tenantId}-${sessionId}`,
  Tags: [{ Key: "tenant_id", Value: tenantId }],
  DurationSeconds: 3600,
});
await ecs.runTask({
  capacityProviderStrategy: [
    { capacityProvider: "FARGATE_SPOT", weight: 4 },
    { capacityProvider: "FARGATE", weight: 1, base: 0 },
  ],
  overrides: {
    cpu: "1024",
    memory: "2048",
    ephemeralStorage: { sizeInGiB: 21 },
  },
  tags: [
    { key: "tenant_id", value: tenantId },
    { key: "session_id", value: sessionId },
  ],
});

✅ Tenant tag on session; S3 paths scoped.
❌ Shared task role with Resource: * and hope.

Bootstrap the agent with short-lived credentials from the orchestrator over the task ENI — do not rely on a long-lived shared task role that can reach every tenant bucket.

Noisy-neighbor controls

Emit per-tenant CloudWatch metrics (RunningTaskCount, BedrockInvocations, EphemeralBytes). Alarm when one tenant consumes >N% of fleet. Cost attribution mirrors cross-account Bedrock patterns.

// admission control
async function admit(tenantId: string) {
  const running = await countTasks(tenantId);
  if (running >= softLimit(tenantId)) await enqueue(tenantId);
  if (running >= hardLimit(tenantId)) throw new Error("tenant_concurrency_exhausted");
}

Spot interruption handling

// listen for Spot interruption notice via container metadata / EventBridge
async function onSpotInterrupt(taskArn: string) {
  await checkpointSession(taskArn); // prompts + tool log to S3
  await rescheduleOnDemand(taskArn); // paid tier: Fargate base
}

Checkpoint format should be replayable: messages, tool results, git SHA, and env digests. Deterministic replay of interrupted sessions: Deterministic Replay.

Network and secret boundaries

• Private subnets only; no public IP on agent tasks
• VPC endpoints for S3, ECR, Bedrock, STS, Logs
• Domain allowlist proxy for github.com / package registries — deny everything else
• Disable IMDS for the app container (httpHopLimit / task role only) so a prompt-injected agent cannot steal credentials via 169.254.169.254

Secrets for package tokens: inject via SSM/Secrets Manager at task start scoped by tenant tag, never bake into the image.

Observability and abuse signals

Watch for:

• Sudden clone of huge monorepos (egress + disk)
• Tool loops calling the same shell command hundreds of times
• Cross-tenant S3 AccessDenied spikes (confused deputy attempts)

Feed those into the same ChatOps draft flow from Incident ChatOps AI — draft tickets, don’t auto-kill without policy.

Cost and quota attribution

Tag every Bedrock invoke, ECS task, and S3 put with tenant_id. Export CUR/Cost Explorer splits weekly. When a tenant approaches their included agent-hours, degrade to smaller models or queue rather than stealing Spot capacity from neighbors. Publish a tenant-facing usage meter so finance conversations are not forensic archaeology.

bedrock.invoke({
  modelId,
  // ...
  // request metadata / logging fields
  trace: { tenantId, sessionId, surface: "coding-agent" },
});

Pair quotas with LLM Cost Controls so per-engineer budgets still apply inside a tenant.

Closing checklist

• [ ] Per-tenant IAM via session tags; no shared writable buckets
• [ ] Fargate Spot + on-demand fallback by tier
• [ ] Concurrency and Bedrock budgets per tenant
• [ ] Egress allowlist; no IMDS from app container (hop limit / task ENI)
• [ ] Checkpoint on Spot interrupt
• [ ] Cell/bulkhead story documented for enterprise tenants
• [ ] Admission control soft/hard limits enforced in orchestrator
• [ ] Ephemeral disk + tmp cleaner; reject oversized clones

Related reading

• Secure AI Sandboxes: Ephemeral ECS Tasks for Agent Tool Execution
• Bedrock Retrieval Filters: Tenant Isolation for Multi-SaaS Code RAG
• Cross-Account Bedrock Access: Platform Teams Without Shared Keys
• LLM Cost Controls: Token Budgets Per PR and Per Engineer

Last updated on September 11, 2026