LLM Cost Controls: Token Budgets Per PR and Per Engineer

One enthusiastic engineer + an agentic refactor loop can torch the monthly Bedrock budget before lunch. Cost control is not a finance spreadsheet after the invoice — it is runtime enforcement keyed by engineer, PR, and service. This guide wires usage plans, application inference profiles, and burn alerts so product traffic keeps flowing while runaway agents get throttled.

⚡ TL;DR: Attribute every InvokeModel to engineer, pr, and service tags. Enforce soft budgets in your gateway and hard budgets with Bedrock application inference profiles / account quotas. Alert Slack at 50/80/100% burn. Kill switches beat surprise invoices. Illustrative default: 2M input tokens/engineer/day for IDE proxies; 500k tokens/PR for CI agents unless elevated.

Attribution first — or you cannot budget

// middleware that stamps identity before Bedrock proxy
export function budgetContext(req: {
  engineerId: string;
  prNumber?: number;
  service: string;
}) {
  return {
    tags: {
      engineer: req.engineerId,
      pr: req.prNumber ? String(req.prNumber) : "none",
      service: req.service,
    },
  };
}

Emit CloudWatch EMF metrics on every call:

{
  "_aws": {
    "Timestamp": 1789100000000,
    "CloudWatchMetrics": [{
      "Namespace": "CheatCoders/LLM",
      "Dimensions": [["engineer"], ["service"], ["pr"]],
      "Metrics": [
        {"Name": "InputTokens", "Unit": "Count"},
        {"Name": "OutputTokens", "Unit": "Count"},
        {"Name": "EstimatedUsd", "Unit": "None"}
      ]
    }]
  },
  "engineer": "aisha",
  "service": "cursor-proxy",
  "pr": "1842",
  "InputTokens": 12000,
  "OutputTokens": 1800,
  "EstimatedUsd": 0.087
}

Dual gates: gateway soft limits + Bedrock hard limits

Soft (product UX): API Gateway usage plans or your Node proxy returns 429 with Retry-After when the engineer’s daily token counter exceeds quota.

Hard (billing fence): Bedrock application inference profiles / provisioned ceilings / account Service Quotas so even a bug that bypasses the proxy cannot free-fire.

# debit_budget.py — DynamoDB conditional debit
import boto3, time
ddb = boto3.resource("dynamodb").Table("LlmBudgets")

def debit(engineer: str, tokens: int, day: str) -> None:
    ddb.update_item(
        Key={"pk": f"ENG#{engineer}", "sk": f"DAY#{day}"},
        UpdateExpression="ADD tokens :t SET updatedAt = :u",
        ConditionExpression="attribute_not_exists(tokens) OR tokens < :max",
        ExpressionAttributeValues={":t": tokens, ":max": 2_000_000, ":u": int(time.time())},
    )

# Illustrative — Slack burn alert via Chatbot / SNS
aws cloudwatch put-metric-alarm \
  --alarm-name llm-org-80pct \
  --namespace CheatCoders/LLM \
  --metric-name EstimatedUsd \
  --threshold 800 \
  --comparison-operator GreaterThanThreshold \
  --period 86400 --evaluation-periods 1 --statistic Sum \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:llm-burn

Per-PR budgets for CI agents

CI agents are worse than IDEs: they retry. Bind a budget to repo+pr:

Post a sticky PR comment when the agent hits 80% so humans can elevate consciously. Pair with AI Code Review Bots least privilege so cost and IAM rise together.

Route cheap vs expensive models

Cost control is also routing: drafts on small models, verifiers on frontier — see multi-model routing patterns. Batch overnight refactors on Batch Inference (companion) instead of interactive Sonnet loops.

function pickModel(task: "draft" | "review" | "security"): string {
  switch (task) {
    case "draft":
      return process.env.MODEL_DRAFT!; // cheap
    case "review":
      return process.env.MODEL_REVIEW!;
    case "security":
      return process.env.MODEL_FRONTIER!; // rare
  }
}

Kill switches

Keep a feature flag llm.proxy.enabled and a per-service flag. When org burn hits 100%, flip flags before you scramble IAM. Document the break-glass elevation path (who can raise quotas, for how long).

Closing checklist

✅ Dos
– ✅ Tag every invoke with engineer, PR, service
– ✅ Enforce soft 429s and hard Bedrock ceilings
– ✅ Slack alert at 50/80/100% org burn
– ✅ Separate IDE vs CI vs production product budgets
– ✅ Prefer batch + small models for mechanical work

❌ Don’ts
– ❌ Don’t share one API key across the company with no attribution
– ❌ Don’t rely on monthly invoice surprise as your control loop
– ❌ Don’t let CI retry storms ignore PR budgets
– ❌ Don’t give developers console rights to raise account quotas casually
– ❌ Don’t skip EstimatedUsd metrics because “tokens are enough”

Related reading

• Bedrock Prompt Caching and Batch Inference: Cut Latency and Cost
• AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines
• Amazon Bedrock Agents: Tool Use, Memory, and Production Guardrails
• AppSync With Bedrock: GraphQL Resolvers That Call Tools Safely (companion in this series)

Last updated on September 11, 2026