18 views
You turned on 5% log sampling to cut CloudWatch Logs ingest. Overnight, tenant cost dashboards went flat while Bedrock bills climbed — because token usage lived only in sampled JSON logs. The unfair advantage is Embedded Metric Format (EMF): metrics extracted by the CloudWatch agent / Lambda Logs subscription before your sampling filter drops the pretty debug lines, with dimensions that make per-tenant unit economics real.
⚡ TL;DR: Emit EMF blobs (_aws+ metrics) on every model call withTenantId,ModelId,Operation. Put counters forInputTokens,OutputTokens,Invocations,EstimatedCostUsd. Keep verbose prompt logs sampled/redacted separately. Pair with Lambda Log Buffering, Cost-Aware RAG Caches, and OpenTelemetry Sampling for Node (metrics ≠ traces).
CloudWatch EMF for LLM Cost: production guidance
typescript
When a subscription filter or Lambda extension samples at 5%, 95% of those lines never become Logs Insights rows. Metrics from EMF are not subject to that same application-log sample rate if you emit them as dedicated metric documents (or ensure the EMF line is always written unsampled).
// ❌ Token usage only in application logs — sampling deletes the evidence
console.log(
JSON.stringify({
msg: "bedrock_done",
tenantId,
inputTokens: usage.inputTokens,
outputTokens: usage.outputTokens,
})
);
Emit EMF from Node Lambda (Powertools or manual)
Manual EMF (no dependency) — one JSON line to stdout:typescript
With AWS Lambda Powertools Metrics (same idea):
type EmfDims = {
TenantId: string;
ModelId: string;
Operation: "Converse" | "ConverseStream" | "Embed" | "Retrieve";
Environment: string;
};
export function emitLlmCostMetrics(
dims: EmfDims,
values: {
inputTokens: number;
outputTokens: number;
latencyMs: number;
estimatedCostUsd: number;
}
) {
const namespace = "CheatCoders/LLM";
const emf = {
_aws: {
Timestamp: Date.now(),
CloudWatchMetrics: [
{
Namespace: namespace,
Dimensions: [
["TenantId", "ModelId", "Operation", "Environment"],
["ModelId", "Operation", "Environment"], // rollup without tenant cardinality explosion in alarms
["Environment"],
],
Metrics: [
{ Name: "InputTokens", Unit: "Count" },
{ Name: "OutputTokens", Unit: "Count" },
{ Name: "Invocations", Unit: "Count" },
{ Name: "LatencyMs", Unit: "Milliseconds" },
{ Name: "EstimatedCostUsd", Unit: "None" },
],
},
],
},
...dims,
InputTokens: values.inputTokens,
OutputTokens: values.outputTokens,
Invocations: 1,
LatencyMs: values.latencyMs,
EstimatedCostUsd: values.estimatedCostUsd,
};
// ✅ Always print EMF — do not put behind sampled logger
process.stdout.write(JSON.stringify(emf) + "\n");
}
typescript
Hook it next to Bedrock:
import { Metrics, MetricUnit } from "@aws-lambda-powertools/metrics";
const metrics = new Metrics({
namespace: "CheatCoders/LLM",
serviceName: "coding-gateway",
});
export function recordUsage(opts: {
tenantId: string;
modelId: string;
inputTokens: number;
outputTokens: number;
costUsd: number;
}) {
metrics.addDimension("TenantId", opts.tenantId);
metrics.addDimension("ModelId", opts.modelId);
metrics.addDimension("Operation", "Converse");
metrics.addDimension("Environment", process.env.STAGE ?? "dev");
metrics.addMetric("InputTokens", MetricUnit.Count, opts.inputTokens);
metrics.addMetric("OutputTokens", MetricUnit.Count, opts.outputTokens);
metrics.addMetric("Invocations", MetricUnit.Count, 1);
metrics.addMetric("EstimatedCostUsd", MetricUnit.None, opts.costUsd);
metrics.publishStoredMetrics(); // flushes EMF
}
typescript
const t0 = Date.now();
const out = await bedrock.send(new ConverseCommand({ /* ... */ }));
const usage = out.usage ?? { inputTokens: 0, outputTokens: 0 };
const cost =
(usage.inputTokens ?? 0) * PRICE_IN + (usage.outputTokens ?? 0) * PRICE_OUT;
emitLlmCostMetrics(
{
TenantId: tenantId,
ModelId: modelId,
Operation: "Converse",
Environment: process.env.STAGE ?? "dev",
},
{
inputTokens: usage.inputTokens ?? 0,
outputTokens: usage.outputTokens ?? 0,
latencyMs: Date.now() - t0,
estimatedCostUsd: cost,
}
);
Cardinality and sampling coexistence
High cardinality warning:TenantId as a dimension is powerful and expensive if you have 100k tenants. Patterns that work:
- EMF with tenant dim for Metrics where you need chargeback (accept cost or use Contributor Insights).
- Separate rollup dimension sets without
TenantIdfor fleet SLOs. - For ultra-high tenant counts, emit tenant id as a property (not dimension) and use Logs Insights / metric filters only for top-N, or aggregate in your own billing table.
typescript
Buffering: if you batch logs, flush EMF immediately — Lambda Log Buffering — never lose the metric line in a crashed buffer.
// ✅ Split planes
emitLlmCostMetrics(...); // always
if (Math.random() < 0.05) {
logger.debug("prompt_sample", { tenantId, promptHash, chars: prompt.length });
}
Alarms and unit economics
typescript
Practical checks:
// Alarm idea: estimated cost spike per environment (rollup dims)
// CloudWatch: SUM(EstimatedCostUsd) > budget for 5m
// Plus: SUM(InputTokens) anomaly on ModelId=claude-3-5 for runaway agents
- Cost per successful PR / per resolved ticket =
SUM(EstimatedCostUsd) / business_counter - Cache hit savings — emit a parallel metric when Cost-Aware RAG Caches skip Bedrock
- Stream vs buffer — tag
Operation=ConverseStreamso TTFB work in the streaming post does not hide token burn
Pricing table next to the emitter
Hard-coding prices in three services guarantees drift. Keep a small versioned map loaded from SSM or a JSON baked at deploy:typescript
When AWS changes list prices, bump the map in one PR — EMF export const BEDROCK_PRICE_PER_1K: Record<string, { in: number; out: number }> = {
"anthropic.claude-3-5-sonnet-20241022-v2:0": { in: 0.003, out: 0.015 },
"amazon.titan-embed-text-v2:0": { in: 0.0002, out: 0 },
};
export function estimateUsd(modelId: string, inputTokens: number, outputTokens: number) {
const p = BEDROCK_PRICE_PER_1K[modelId] ?? { in: 0, out: 0 };
return (inputTokens / 1000) * p.in + (outputTokens / 1000) * p.out;
}
EstimatedCostUsd stays comparable across dashboards. For chargeback, optionally write the same row to DynamoDB/Timestream keyed by day+tenant so finance does not scrape CloudWatch.
Verify EMF actually becomes metrics
After deploy, invoke once and confirm in CloudWatch Metrics underCheatCoders/LLM. If the namespace is empty, you usually have: invalid EMF JSON (trailing commas), dimensions array mismatch with property names, or a log router dropping non-sampled lines incorrectly. Fix before trusting any budget alarm.
Checklist
- [ ] EMF (or Powertools Metrics) on every billable model/tool path
- [ ] Dimensions: Environment + ModelId + Operation; TenantId only if cardinality OK
- [ ]
InputTokens/OutputTokens/Invocations/EstimatedCostUsdalways unsampled - [ ] Verbose prompt logs sampled + redacted on a different path
- [ ] Rollup alarms on non-tenant dimension sets
- [ ] Price table versioned next to emitter (model price changes)
- [ ] Dashboards: cost by model, top tenants (Contributor Insights or billing ETL)
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- REST API Design Best Practices: The Patterns That Make APIs a Joy to Use
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
