Lambda Plus Bedrock: Stream Tokens Without API Gateway Caps

API Gateway REST/HTTP APIs are a poor fit for long token streams: idle timeouts, payload caps, and buffering habits fight TTFB. Lambda Function URLs with RESPONSE_STREAM invoke mode plus Bedrock ConverseStream / InvokeModelWithResponseStream let you flush tokens as they arrive — if you honor backpressure, client aborts, heartbeats, and memory bounds. Buffering the whole completion in RAM is how 3k-token answers OOM a 512 MB function.

⚡ TL;DR: Use Function URL + AWS_LAMBDA_RESPONSE_STREAMING; pipe Bedrock stream chunks to awslambda.streamifyResponse / Node HttpResponseStream; abort Bedrock when req closes; send heartbeat comments every ~10s; cap max tokens and max wall time. See Lambda cold starts Node 20, Lambda timeouts/DLQs, Powertools structured logs.

Function URL streaming handler (Node 20)

import {
  BedrockRuntimeClient,
  ConverseStreamCommand,
} from "@aws-sdk/client-bedrock-runtime";
// @ts-expect-error provided by Lambda Node streaming runtime
import { streamifyResponse } from "aws-lambda/stream";

const bedrock = new BedrockRuntimeClient({});

export const handler = streamifyResponse(async (event, responseStream, context) => {
  const prompt = JSON.parse(event.body ?? "{}").prompt as string;
  if (!prompt || prompt.length > 12_000) {
    responseStream.write(JSON.stringify({ error: "bad_request" }));
    responseStream.end();
    return;
  }

  const httpStream = awslambda.HttpResponseStream.from(responseStream, {
    statusCode: 200,
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
    },
  });

  const ac = new AbortController();
  // ✅ Abort Bedrock when the client disconnects / Lambda times out budget
  context = context; // keep soft deadline
  const softMs = Math.max(1_000, (context.getRemainingTimeInMillis?.() ?? 30_000) - 2_000);
  const timer = setTimeout(() => ac.abort(), softMs);

  try {
    const stream = await bedrock.send(
      new ConverseStreamCommand({
        modelId: process.env.MODEL_ID!,
        messages: [{ role: "user", content: [{ text: prompt }] }],
        inferenceConfig: { maxTokens: 2048, temperature: 0.2 },
      }),
      { abortSignal: ac.signal }
    );

    let lastBeat = Date.now();
    for await (const ev of stream.stream ?? []) {
      if (Date.now() - lastBeat > 10_000) {
        httpStream.write(": heartbeat\n\n"); // ✅ keep intermediaries awake
        lastBeat = Date.now();
      }
      const text = ev.contentBlockDelta?.delta?.text;
      if (text) {
        // ✅ Write incrementally — never accumulate full answer
        httpStream.write(`data: ${JSON.stringify({ text })}\n\n`);
      }
    }
    httpStream.write(`data: ${JSON.stringify({ done: true })}\n\n`);
  } catch (err: any) {
    if (err?.name !== "AbortError") {
      httpStream.write(`data: ${JSON.stringify({ error: "upstream" })}\n\n`);
    }
  } finally {
    clearTimeout(timer);
    httpStream.end();
  }
});

// ❌ Wrong: buffer entire completion then return
const chunks: string[] = [];
for await (const ev of stream) chunks.push(ev.text);
return { statusCode: 200, body: chunks.join("") };

Backpressure and memory bounds

Node streams will buffer if the client is slow. Prefer writing small SSE frames and avoid holding chunk arrays. Set function memory so the runtime has headroom for the SDK, not the answer text (illustrative: 1024 MB for ConverseStream, not because you store tokens).

Auth and cost controls on Function URLs

Function URLs should not be anonymous on production coding tools. Use AWS_IAM auth or a JWT authorizer at a thin CloudFront/Lambda@Edge layer. Emit token usage metrics with Powertools (structured logs).

// ✅ After stream: log usage without prompt text
logger.info("bedrock.stream.complete", {
  inputTokens: usage?.inputTokens,
  outputTokens: usage?.outputTokens,
  modelId: process.env.MODEL_ID,
});

When API Gateway is still OK

Short completions (< ~30s), non-streaming JSON, or WebSocket APIs with your own frame protocol. For IDE-like token streaming, Function URLs (or ALB + streaming container) win. Timeouts/retries for async siblings remain relevant (Lambda timeouts).

Closing checklist

✅ Dos
– ✅ Function URL + response streaming + Bedrock stream APIs
– ✅ Abort upstream on client/deadline
– ✅ Heartbeat SSE; cap maxTokens
– ✅ IAM or signed URL auth on the Function URL
– ✅ Log token usage, not raw prompts

❌ Don’ts
– ❌ Don’t buffer full completions in memory
– ❌ Don’t put long SSE through API Gateway REST
– ❌ Don’t leave Function URLs AuthType NONE in prod
– ❌ Don’t ignore soft deadline vs Lambda timeout
– ❌ Don’t stream secrets — attach Guardrails (agents/guardrails)

Related reading

• Lambda Cold Starts on Node 20: Measure, Cut, and Keep Cutting
• Lambda Timeouts, Retries, and DLQs: Idempotent Failure Handling
• Lambda Powertools for Node: Structured Logs That Survive On-Call
• Amazon Bedrock Agents: Tool Use, Memory, and Production Guardrails

Last updated on September 11, 2026