CloudWatch EMF for LLM Cost: Per-Tenant Token Metrics That Survive Sampling

You turned on 5% log sampling to cut CloudWatch Logs ingest. Overnight, tenant cost dashboards went flat while Bedrock bills climbed — because token usage lived only in sampled JSON logs. The unfair advantage is Embedded Metric Format (EMF): metrics extracted by the CloudWatch agent / Lambda Logs subscription before your sampling filter drops the pretty debug lines, with dimensions that make per-tenant unit economics real.

⚡ TL;DR: Emit EMF blobs (_aws + metrics) on every model call with TenantId, ModelId, Operation. Put counters for InputTokens, OutputTokens, Invocations, EstimatedCostUsd. Keep verbose prompt logs sampled/redacted separately. Pair with Lambda Log Buffering, Cost-Aware RAG Caches, and OpenTelemetry Sampling for Node (metrics ≠ traces).

Why sampled logs cannot be your cost system of record

// ❌ Token usage only in application logs — sampling deletes the evidence
console.log(
  JSON.stringify({
    msg: "bedrock_done",
    tenantId,
    inputTokens: usage.inputTokens,
    outputTokens: usage.outputTokens,
  })
);

When a subscription filter or Lambda extension samples at 5%, 95% of those lines never become Logs Insights rows. Metrics from EMF are not subject to that same application-log sample rate if you emit them as dedicated metric documents (or ensure the EMF line is always written unsampled).

Emit EMF from Node Lambda (Powertools or manual)

Manual EMF (no dependency) — one JSON line to stdout:

type EmfDims = {
  TenantId: string;
  ModelId: string;
  Operation: "Converse" | "ConverseStream" | "Embed" | "Retrieve";
  Environment: string;
};

export function emitLlmCostMetrics(
  dims: EmfDims,
  values: {
    inputTokens: number;
    outputTokens: number;
    latencyMs: number;
    estimatedCostUsd: number;
  }
) {
  const namespace = "CheatCoders/LLM";
  const emf = {
    _aws: {
      Timestamp: Date.now(),
      CloudWatchMetrics: [
        {
          Namespace: namespace,
          Dimensions: [
            ["TenantId", "ModelId", "Operation", "Environment"],
            ["ModelId", "Operation", "Environment"], // rollup without tenant cardinality explosion in alarms
            ["Environment"],
          ],
          Metrics: [
            { Name: "InputTokens", Unit: "Count" },
            { Name: "OutputTokens", Unit: "Count" },
            { Name: "Invocations", Unit: "Count" },
            { Name: "LatencyMs", Unit: "Milliseconds" },
            { Name: "EstimatedCostUsd", Unit: "None" },
          ],
        },
      ],
    },
    ...dims,
    InputTokens: values.inputTokens,
    OutputTokens: values.outputTokens,
    Invocations: 1,
    LatencyMs: values.latencyMs,
    EstimatedCostUsd: values.estimatedCostUsd,
  };
  // ✅ Always print EMF — do not put behind sampled logger
  process.stdout.write(JSON.stringify(emf) + "\n");
}

With AWS Lambda Powertools Metrics (same idea):

import { Metrics, MetricUnit } from "@aws-lambda-powertools/metrics";

const metrics = new Metrics({
  namespace: "CheatCoders/LLM",
  serviceName: "coding-gateway",
});

export function recordUsage(opts: {
  tenantId: string;
  modelId: string;
  inputTokens: number;
  outputTokens: number;
  costUsd: number;
}) {
  metrics.addDimension("TenantId", opts.tenantId);
  metrics.addDimension("ModelId", opts.modelId);
  metrics.addDimension("Operation", "Converse");
  metrics.addDimension("Environment", process.env.STAGE ?? "dev");
  metrics.addMetric("InputTokens", MetricUnit.Count, opts.inputTokens);
  metrics.addMetric("OutputTokens", MetricUnit.Count, opts.outputTokens);
  metrics.addMetric("Invocations", MetricUnit.Count, 1);
  metrics.addMetric("EstimatedCostUsd", MetricUnit.None, opts.costUsd);
  metrics.publishStoredMetrics(); // flushes EMF
}

Hook it next to Bedrock:

const t0 = Date.now();
const out = await bedrock.send(new ConverseCommand({ /* ... */ }));
const usage = out.usage ?? { inputTokens: 0, outputTokens: 0 };
const cost =
  (usage.inputTokens ?? 0) * PRICE_IN + (usage.outputTokens ?? 0) * PRICE_OUT;

emitLlmCostMetrics(
  {
    TenantId: tenantId,
    ModelId: modelId,
    Operation: "Converse",
    Environment: process.env.STAGE ?? "dev",
  },
  {
    inputTokens: usage.inputTokens ?? 0,
    outputTokens: usage.outputTokens ?? 0,
    latencyMs: Date.now() - t0,
    estimatedCostUsd: cost,
  }
);

Cardinality and sampling coexistence

High cardinality warning: TenantId as a dimension is powerful and expensive if you have 100k tenants. Patterns that work:

1. EMF with tenant dim for Metrics where you need chargeback (accept cost or use Contributor Insights).
2. Separate rollup dimension sets without TenantId for fleet SLOs.
3. For ultra-high tenant counts, emit tenant id as a property (not dimension) and use Logs Insights / metric filters only for top-N, or aggregate in your own billing table.

Keep prompt bodies on a sampled structured logger with redaction — see Log Injection Defenses and bootcamp-adjacent hygiene without dumping secrets. EMF lines stay unsampled and tiny.

// ✅ Split planes
emitLlmCostMetrics(...); // always
if (Math.random() < 0.05) {
  logger.debug("prompt_sample", { tenantId, promptHash, chars: prompt.length });
}

Buffering: if you batch logs, flush EMF immediately — Lambda Log Buffering — never lose the metric line in a crashed buffer.

Alarms and unit economics

// Alarm idea: estimated cost spike per environment (rollup dims)
// CloudWatch: SUM(EstimatedCostUsd) > budget for 5m
// Plus: SUM(InputTokens) anomaly on ModelId=claude-3-5 for runaway agents

Practical checks:

• Cost per successful PR / per resolved ticket = SUM(EstimatedCostUsd) / business_counter
• Cache hit savings — emit a parallel metric when Cost-Aware RAG Caches skip Bedrock
• Stream vs buffer — tag Operation=ConverseStream so TTFB work in the streaming post does not hide token burn

For agent queue delay vs model latency, keep those as separate metrics (queue wait) so cost EMF stays clean — same spirit as p99 queueing analysis without conflating bills.

Pricing table next to the emitter

Hard-coding prices in three services guarantees drift. Keep a small versioned map loaded from SSM or a JSON baked at deploy:

export const BEDROCK_PRICE_PER_1K: Record<string, { in: number; out: number }> = {
  "anthropic.claude-3-5-sonnet-20241022-v2:0": { in: 0.003, out: 0.015 },
  "amazon.titan-embed-text-v2:0": { in: 0.0002, out: 0 },
};

export function estimateUsd(modelId: string, inputTokens: number, outputTokens: number) {
  const p = BEDROCK_PRICE_PER_1K[modelId] ?? { in: 0, out: 0 };
  return (inputTokens / 1000) * p.in + (outputTokens / 1000) * p.out;
}

When AWS changes list prices, bump the map in one PR — EMF EstimatedCostUsd stays comparable across dashboards. For chargeback, optionally write the same row to DynamoDB/Timestream keyed by day+tenant so finance does not scrape CloudWatch.

Verify EMF actually becomes metrics

After deploy, invoke once and confirm in CloudWatch Metrics under CheatCoders/LLM. If the namespace is empty, you usually have: invalid EMF JSON (trailing commas), dimensions array mismatch with property names, or a log router dropping non-sampled lines incorrectly. Fix before trusting any budget alarm.

Checklist

• [ ] EMF (or Powertools Metrics) on every billable model/tool path
• [ ] Dimensions: Environment + ModelId + Operation; TenantId only if cardinality OK
• [ ] InputTokens / OutputTokens / Invocations / EstimatedCostUsd always unsampled
• [ ] Verbose prompt logs sampled + redacted on a different path
• [ ] Rollup alarms on non-tenant dimension sets
• [ ] Price table versioned next to emitter (model price changes)
• [ ] Dashboards: cost by model, top tenants (Contributor Insights or billing ETL)