AI Architecture Bootcamp Archives - Page 3 of 10

Day 15: Cost Curves: Input Tokens Dominate

Input tokens dominate cost curves — caching, draft models, and per-PR budgets engineers can actually see.

Build a read-only on-call copilot: runbook RAG, suggest-only patches, dual control before any prod mutation.

Install guardrails before clever prompts — Bedrock Guardrails, allowlists, and injection tests on ticket text.

OpenTelemetry spans across prompt build, retrieval, model, and tools so a human can replay an agent failure without guessing.

Scratchpad, session, and long-term memory belong in different stores — stop stuffing DynamoDB into the prompt by default.

ReAct loops, plan-then-act, and compiler-style agents each spend tokens differently — cap iterations before they burn your budget.

Survive a regional Bedrock outage without a broken UX or a split-brain vector index — health checks, sticky sessions, and index strategy.

Avoid Sev-1 Bedrock throttles by partitioning interactive vs batch capacity and pre-warming provisioned throughput for incident copilots.

Treat tool use as a versioned API: OpenAPI schemas, idempotency keys, timeouts, and typed errors — not prompt poetry.

Private Bedrock via VPC endpoints, Secrets Manager rotation agents respect, and zero API keys in prompts or logs.