AWS Lambda Response Streaming: Stream Coding-Agent Tokens Without Buffering Full Completions

27 views
Your coding-agent API calls Bedrock, waits for the whole completion, then returns one JSON blob. For a short answer that’s fine. For a multi-file patch review, the user watches a spinner for a long time, your IDE extension’s idle timeout fires, and the function holds the entire answer in memory before sending a single byte. None of that waiting is the model’s fault. It’s the buffering. Lambda response streaming lets a function send its response in pieces as they become available. Paired with Bedrock’s ConverseStream, the first tokens reach the user as soon as the model produces them. This guide explains how streaming works, which entry point to choose (function URL, API Gateway or the SDK), how to write a handler that streams Bedrock tokens safely, and how to keep cost and timeouts under control. It also absorbs our earlier posts on streaming gateways and Bedrock token streams, so everything is in one place.
TL;DR
  • Set the function URL’s invoke mode to RESPONSE_STREAM. Buffered responses are capped at 6 MB; streamed responses can be up to 200 MB. The first 6 MB stream at full speed, and anything after that is capped at 2 MBps.
  • Native streaming works on the Node.js managed runtimes. Python needs a custom runtime or the Lambda Web Adapter.
  • API Gateway REST APIs can now stream Lambda proxy responses too (response transfer mode STREAM), for up to 15 minutes.
  • Lambda does not stop your function when the client disconnects, and you pay for the full duration. Bound every stream with a soft deadline, maxTokens and a sensible function timeout.

How response streaming works

A normal (buffered) invocation runs your handler to completion, collects the return value, and only then sends it to the caller. With streaming, your handler gets a writable stream. Every write() can reach the client right away, and the response ends when you call end(). A few facts from the Lambda documentation shape every design decision:
  • Two invoke modes. BUFFERED (the default) uses the Invoke API with a 6 MB response limit. RESPONSE_STREAM uses InvokeWithResponseStream with a 200 MB limit.
  • Bandwidth. The first 6 MB of a response has uncapped bandwidth. After that, Lambda streams at up to 2 MBps. Token streams for chat and code review rarely get near 6 MB, so this mostly matters for file downloads.
  • Runtimes. Lambda supports streaming on Node.js managed runtimes. For Python and other languages, use a custom runtime that implements streaming or the Lambda Web Adapter.
  • The console lies (a little). When you test in the Lambda console, responses always appear buffered. Test with curl --no-buffer instead.
  • Disconnects don’t stop you. Streamed responses aren’t interrupted when the client connection breaks, and you’re billed for the full function duration.
Figure 1. Token path from Bedrock to the client, and the three ways to invoke a streaming function. Diagram by CheatCoders.
Figure 1. Token path from Bedrock to the client, and the three ways to invoke a streaming function. Diagram by CheatCoders.

Which entry point should you use?

Entry pointHow it streamsWatch out for
Lambda function URLSet InvokeMode: RESPONSE_STREAM on the URLFunction URLs don’t support response streaming within a VPC environment. Use AWS_IAM auth, not NONE.
API Gateway REST APILambda proxy integration with response transfer mode STREAMREST APIs only. Streams last up to 15 minutes. Idle timeout is 5 minutes for Regional/private endpoints and 30 seconds for edge-optimized. No caching, VTL response transformation or content encoding.
InvokeWithResponseStream from your backendThe SDK returns an event stream of payload chunksYour backend re-streams to the browser. Inside a VPC, this is the documented route (through a Lambda interface endpoint).
If you already run an API Gateway REST API with authorizers, usage plans and WAF, streaming through it is now the simplest option. For a small internal tool, a function URL with IAM auth behind your own backend is less to manage. Either way, the handler code is the same.

Before you start

  • A Node.js 22 or 24 Lambda function. The awslambda global is provided by the runtime, so there’s nothing to import.
  • An execution role that allows bedrock:InvokeModelWithResponseStream (that’s the permission ConverseStream uses) on the model or inference profile you call.
  • Callers that can sign requests with SigV4 (your backend, or curl --aws-sigv4 for testing).

Step 1: Create a streaming function URL with IAM auth

bash
aws lambda create-function-url-config \
  --function-name coding-agent-stream \
  --auth-type AWS_IAM \
  --invoke-mode RESPONSE_STREAM

# Switching an existing URL later:
aws lambda update-function-url-config \
  --function-name coding-agent-stream \
  --invoke-mode RESPONSE_STREAM
In CloudFormation, the same setting is the InvokeMode property on AWS::Lambda::Url:
yaml
CodingAgentStreamUrl:
  Type: AWS::Lambda::Url
  Properties:
    TargetFunctionArn: !GetAtt CodingAgentStream.Arn
    AuthType: AWS_IAM
    InvokeMode: RESPONSE_STREAM
Since October 2025, new function URLs require callers to have both lambda:InvokeFunctionUrl and lambda:InvokeFunction. The lambda:InvokedViaFunctionUrl condition key limits the second permission to calls that come through the URL. Here’s an identity policy for the backend role that calls the URL:
json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "lambda:InvokeFunctionUrl",
      "Resource": "arn:aws:lambda:us-east-1:111122223333:function:coding-agent-stream"
    },
    {
      "Effect": "Allow",
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:us-east-1:111122223333:function:coding-agent-stream",
      "Condition": { "Bool": { "lambda:InvokedViaFunctionUrl": "true" } }
    }
  ]
}

Step 2: Write the streaming handler

The handler below streams Bedrock tokens as newline-delimited JSON (NDJSON), one event per line. That format is easy to parse in browsers, CLIs and IDE extensions. Three details matter more than the rest:
  • Write deltas as they arrive. Never collect the full completion in an array and return it at the end; that’s buffering with extra steps.
  • Stop before the function timeout. A soft deadline aborts the Bedrock call a few seconds before Lambda would kill the function, so the client gets a clean error event instead of a cut-off stream.
  • Don’t let the client choose the model or the token budget. Both come from configuration.
typescript
// index.mjs (Node.js managed runtime). awslambda is a runtime global.
import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";

const bedrock = new BedrockRuntimeClient({});
const MODEL_ID = process.env.MODEL_ID;                 // model or inference profile ID
const MAX_TOKENS = Number(process.env.MAX_TOKENS ?? "4096");
const MAX_PROMPT_CHARS = 60_000;

const send = (stream, event) => stream.write(JSON.stringify(event) + "\n");

// Best effort only: secrets can be split across deltas. Keep them out of model context instead.
const redact = (text) => text.replace(/AKIA[0-9A-Z]{16}/g, "[REDACTED_KEY]");

export const handler = awslambda.streamifyResponse(async (event, responseStream, context) => {
  responseStream = awslambda.HttpResponseStream.from(responseStream, {
    statusCode: 200,
    headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache" },
  });

  let messages;
  try {
    messages = JSON.parse(event.body ?? "{}").messages;
    if (!Array.isArray(messages) || JSON.stringify(messages).length > MAX_PROMPT_CHARS) throw new Error();
  } catch {
    send(responseStream, { type: "error", code: "bad_request" });
    responseStream.end();
    return;
  }

  // Lambda keeps running after a client disconnect, so bound the work ourselves.
  const abort = new AbortController();
  const budgetMs = Math.max(1000, context.getRemainingTimeInMillis() - 3000);
  const deadline = setTimeout(() => abort.abort(), budgetMs);

  send(responseStream, { type: "session.start", schemaVersion: 1, requestId: context.awsRequestId });
  try {
    const response = await bedrock.send(
      new ConverseStreamCommand({ modelId: MODEL_ID, messages, inferenceConfig: { maxTokens: MAX_TOKENS } }),
      { abortSignal: abort.signal }
    );
    for await (const ev of response.stream) {
      if (ev.contentBlockDelta?.delta?.text) {
        send(responseStream, { type: "delta", text: redact(ev.contentBlockDelta.delta.text) });
      } else if (ev.messageStop) {
        send(responseStream, { type: "stop", reason: ev.messageStop.stopReason });
      } else if (ev.metadata?.usage) {
        send(responseStream, { type: "usage", ...ev.metadata.usage });
      }
    }
    send(responseStream, { type: "done" });
  } catch (err) {
    send(responseStream, { type: "error", code: abort.signal.aborted ? "deadline" : (err.name ?? "upstream") });
  } finally {
    clearTimeout(deadline);
    responseStream.end();
  }
});
Once HttpResponseStream.from has sent the 200 status and headers, you can’t change them. Errors after that point have to travel in-band as an error event, which is why the client contract below includes one. On Node.js 24, Lambda no longer waits for unresolved promises after the handler returns or the stream ends, so await anything you start, such as metric flushes.

Step 3: Test it from a terminal

The AWS tutorial uses curl with SigV4 signing. --no-buffer makes curl print each chunk as it arrives, so you can see the stream working:
bash
curl --no-buffer --request POST "https://abcdefghijklm7nop7qrs740abcd.lambda-url.us-east-1.on.aws/" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  --header "x-amz-security-token: $AWS_SESSION_TOKEN" \
  --aws-sigv4 "aws:amz:us-east-1:lambda" \
  --header "content-type: application/json" \
  --data '{"messages":[{"role":"user","content":[{"text":"Explain this stack trace in 3 bullets."}]}]}'
You should see a session.start line, then a series of delta lines, then stop, usage and done. If everything arrives in one burst at the end, jump to the troubleshooting table.

Step 4: Re-stream from your backend

Browsers can’t sign SigV4 requests safely, so most teams put a small backend in front. It authenticates the user, then calls InvokeWithResponseStream and forwards each payload chunk:
typescript
import { LambdaClient, InvokeWithResponseStreamCommand } from "@aws-sdk/client-lambda";

const lambda = new LambdaClient({});

export async function proxyAgentStream(res, payload) {
  const out = await lambda.send(new InvokeWithResponseStreamCommand({
    FunctionName: "coding-agent-stream",
    Payload: Buffer.from(JSON.stringify({ body: JSON.stringify(payload) })),
  }));
  res.setHeader("Content-Type", "application/x-ndjson");
  for await (const event of out.EventStream) {
    if (event.PayloadChunk?.Payload) res.write(event.PayloadChunk.Payload);
    if (event.InvokeComplete?.ErrorCode) {
      res.write(JSON.stringify({ type: "error", code: event.InvokeComplete.ErrorCode }) + "\n");
    }
  }
  res.end();
}
Notice that the payload wraps the request in a body field. The handler reads event.body because function URLs deliver an HTTP-style event, so a direct SDK invoke needs to imitate that shape (or the handler needs to accept both).

A client contract your IDE plugins can rely on

Streams get consumed by more than one client: the web app, the IDE extension, the CLI. Write the event types down and version them, so you can add fields without breaking old clients:
EventWhenFields
session.startFirst line, alwaysschemaVersion, requestId
deltaEach text chunk from the modeltext
pingWhile a tool runs and no tokens flowts
stop / usageModel finishedreason; token counts
done / errorLast line, exactly one of themcode for errors
Pings matter when something between you and the client enforces an idle timeout, such as API Gateway’s 5-minute idle timeout on Regional REST endpoints. If a tool call takes longer than a few minutes (a full test suite, a CDK synth), don’t hold the stream open at all. Start the work asynchronously and stream a job ID the client can poll or subscribe to.

Cost, cancellation and observability

Because a disconnected client doesn’t stop the function, your controls have to live inside the function: maxTokens on every request, the soft deadline above, and a function timeout that matches the longest answer you’re willing to pay for. For metrics, record time to first byte (TTFB) when the first delta is written, plus token usage from the usage event. A one-line Embedded Metric Format log turns that into a CloudWatch metric without an extra API call:
typescript
// Call once, when the first delta is written.
function recordTtfb(startedAt, stage) {
  console.log(JSON.stringify({
    _aws: {
      Timestamp: Date.now(),
      CloudWatchMetrics: [{ Namespace: "CodingAgent/Stream", Dimensions: [["Stage"]],
        Metrics: [{ Name: "TimeToFirstByteMs", Unit: "Milliseconds" }] }],
    },
    Stage: stage,
    TimeToFirstByteMs: Date.now() - startedAt,
  }));
}

Worked example: streaming a diff review

  1. The IDE extension sends a diff and the question “anything risky here?” to your backend, which checks the user’s session.
  2. The backend calls InvokeWithResponseStream. The handler sends session.start immediately, so the extension can show “reviewing…” with a request ID.
  3. Bedrock starts producing text. Each delta line is painted in the review panel as it arrives, so the engineer starts reading the first findings while the model is still writing.
  4. The model finishes. The handler sends stop (reason end_turn), a usage line for billing, then done.
  5. On another day, a huge diff makes the review run long. Three seconds before the function timeout, the soft deadline aborts the Bedrock call, and the extension receives {"type":"error","code":"deadline"} and suggests splitting the diff, instead of hanging.

Troubleshooting

SymptomLikely causeFix
Whole response arrives at onceURL still in BUFFERED mode, or you’re testing in the Lambda consoleSet RESPONSE_STREAM; test with curl --no-buffer
awslambda is not definedRunning outside the Lambda Node.js runtime (for example, locally)Stub the global in local tests, or test against the deployed function
Python handler never streamsPython managed runtimes don’t support native streamingUse Node.js, a custom runtime, or the Lambda Web Adapter
403 from the function URLCaller lacks lambda:InvokeFunctionUrl or lambda:InvokeFunction, or SigV4 signed for the wrong serviceGrant both actions; sign for service lambda
Streaming fails for a function in a VPCFunction URLs don’t support response streaming within a VPC environmentInvoke with InvokeWithResponseStream through a Lambda interface endpoint
API Gateway still buffersIntegration transfer mode is BUFFERED, or it’s an HTTP APIUse a REST API with an AWS_PROXY integration and STREAM mode
Stream cuts off mid-answerFunction timeout reached, or an idle timeout upstreamSoft deadline before timeout; send ping during tool waits
Large downloads slow down after a whileBandwidth is capped at 2 MBps after the first 6 MBExpected; serve big files from S3 instead

Production checklist

  • ☐ Invoke mode RESPONSE_STREAM (function URL) or STREAM (API Gateway REST)
  • ☐ AWS_IAM auth; callers have both invoke permissions; CORS limited to your origins
  • ☐ Handler writes deltas immediately and always ends with done or error
  • ☐ Model ID and maxTokens come from config, never from the request
  • ☐ Soft deadline a few seconds before the function timeout
  • ☐ Client contract documented and versioned with schemaVersion
  • ☐ TTFB and token usage emitted as metrics; errors alarmed
  • ☐ Long tool runs moved off the stream to an async job

FAQ

Can API Gateway stream Lambda responses now?

Yes, REST APIs can. Set the response transfer mode of a Lambda proxy (AWS_PROXY) integration to STREAM. Streams can run for up to 15 minutes, and the first 10 MB aren’t bandwidth-limited. HTTP APIs aren’t covered by this feature.

Is streaming a replacement for WebSockets?

No. Streaming is one request with a long, incremental response. WebSockets fit when the server needs to push events after the request ends, or when the client sends messages mid-session, such as “stop” or a follow-up.

Does streaming cost more?

You pay for function duration either way, and a stream runs as long as the model takes, just as a buffered call would. The difference is that the user sees output sooner. Check the Lambda and API Gateway pricing pages, since API Gateway charges for response streaming.

How do tool calls fit into a stream?

Emit a tool.start event, run the tool, send pings if it takes a while, emit tool.end, then start a new ConverseStream call with the tool result. Anything longer than a few minutes belongs in an async job.

Official documentation

Related on CheatCoders: Lambda SnapStart · PrivateLink for Bedrock Last updated on October 6, 2026

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a comment

No account needed. Name and email are optional.