27 views
Your coding-agent API calls Bedrock, waits for the whole completion, then returns one JSON blob. For a short answer that’s fine. For a multi-file patch review, the user watches a spinner for a long time, your IDE extension’s idle timeout fires, and the function holds the entire answer in memory before sending a single byte. None of that waiting is the model’s fault. It’s the buffering.
Lambda response streaming lets a function send its response in pieces as they become available. Paired with Bedrock’s ConverseStream, the first tokens reach the user as soon as the model produces them. This guide explains how streaming works, which entry point to choose (function URL, API Gateway or the SDK), how to write a handler that streams Bedrock tokens safely, and how to keep cost and timeouts under control. It also absorbs our earlier posts on streaming gateways and Bedrock token streams, so everything is in one place.
TL;DR
- Set the function URL’s invoke mode to
RESPONSE_STREAM. Buffered responses are capped at 6 MB; streamed responses can be up to 200 MB. The first 6 MB stream at full speed, and anything after that is capped at 2 MBps.- Native streaming works on the Node.js managed runtimes. Python needs a custom runtime or the Lambda Web Adapter.
- API Gateway REST APIs can now stream Lambda proxy responses too (response transfer mode
STREAM), for up to 15 minutes.- Lambda does not stop your function when the client disconnects, and you pay for the full duration. Bound every stream with a soft deadline,
maxTokensand a sensible function timeout.
How response streaming works
A normal (buffered) invocation runs your handler to completion, collects the return value, and only then sends it to the caller. With streaming, your handler gets a writable stream. Everywrite() can reach the client right away, and the response ends when you call end().
A few facts from the Lambda documentation shape every design decision:
- Two invoke modes.
BUFFERED(the default) uses theInvokeAPI with a 6 MB response limit.RESPONSE_STREAMusesInvokeWithResponseStreamwith a 200 MB limit. - Bandwidth. The first 6 MB of a response has uncapped bandwidth. After that, Lambda streams at up to 2 MBps. Token streams for chat and code review rarely get near 6 MB, so this mostly matters for file downloads.
- Runtimes. Lambda supports streaming on Node.js managed runtimes. For Python and other languages, use a custom runtime that implements streaming or the Lambda Web Adapter.
- The console lies (a little). When you test in the Lambda console, responses always appear buffered. Test with
curl --no-bufferinstead. - Disconnects don’t stop you. Streamed responses aren’t interrupted when the client connection breaks, and you’re billed for the full function duration.
Which entry point should you use?
| Entry point | How it streams | Watch out for |
|---|---|---|
| Lambda function URL | Set InvokeMode: RESPONSE_STREAM on the URL | Function URLs don’t support response streaming within a VPC environment. Use AWS_IAM auth, not NONE. |
| API Gateway REST API | Lambda proxy integration with response transfer mode STREAM | REST APIs only. Streams last up to 15 minutes. Idle timeout is 5 minutes for Regional/private endpoints and 30 seconds for edge-optimized. No caching, VTL response transformation or content encoding. |
InvokeWithResponseStream from your backend | The SDK returns an event stream of payload chunks | Your backend re-streams to the browser. Inside a VPC, this is the documented route (through a Lambda interface endpoint). |
Before you start
- A Node.js 22 or 24 Lambda function. The
awslambdaglobal is provided by the runtime, so there’s nothing to import. - An execution role that allows
bedrock:InvokeModelWithResponseStream(that’s the permission ConverseStream uses) on the model or inference profile you call. - Callers that can sign requests with SigV4 (your backend, or
curl --aws-sigv4for testing).
Step 1: Create a streaming function URL with IAM auth
bash
In CloudFormation, the same setting is the aws lambda create-function-url-config \
--function-name coding-agent-stream \
--auth-type AWS_IAM \
--invoke-mode RESPONSE_STREAM
# Switching an existing URL later:
aws lambda update-function-url-config \
--function-name coding-agent-stream \
--invoke-mode RESPONSE_STREAMInvokeMode property on AWS::Lambda::Url:
yaml
Since October 2025, new function URLs require callers to have both CodingAgentStreamUrl:
Type: AWS::Lambda::Url
Properties:
TargetFunctionArn: !GetAtt CodingAgentStream.Arn
AuthType: AWS_IAM
InvokeMode: RESPONSE_STREAMlambda:InvokeFunctionUrl and lambda:InvokeFunction. The lambda:InvokedViaFunctionUrl condition key limits the second permission to calls that come through the URL. Here’s an identity policy for the backend role that calls the URL:
json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "lambda:InvokeFunctionUrl",
"Resource": "arn:aws:lambda:us-east-1:111122223333:function:coding-agent-stream"
},
{
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:111122223333:function:coding-agent-stream",
"Condition": { "Bool": { "lambda:InvokedViaFunctionUrl": "true" } }
}
]
}Step 2: Write the streaming handler
The handler below streams Bedrock tokens as newline-delimited JSON (NDJSON), one event per line. That format is easy to parse in browsers, CLIs and IDE extensions. Three details matter more than the rest:- Write deltas as they arrive. Never collect the full completion in an array and return it at the end; that’s buffering with extra steps.
- Stop before the function timeout. A soft deadline aborts the Bedrock call a few seconds before Lambda would kill the function, so the client gets a clean
errorevent instead of a cut-off stream. - Don’t let the client choose the model or the token budget. Both come from configuration.
typescript
Once // index.mjs (Node.js managed runtime). awslambda is a runtime global.
import { BedrockRuntimeClient, ConverseStreamCommand } from "@aws-sdk/client-bedrock-runtime";
const bedrock = new BedrockRuntimeClient({});
const MODEL_ID = process.env.MODEL_ID; // model or inference profile ID
const MAX_TOKENS = Number(process.env.MAX_TOKENS ?? "4096");
const MAX_PROMPT_CHARS = 60_000;
const send = (stream, event) => stream.write(JSON.stringify(event) + "\n");
// Best effort only: secrets can be split across deltas. Keep them out of model context instead.
const redact = (text) => text.replace(/AKIA[0-9A-Z]{16}/g, "[REDACTED_KEY]");
export const handler = awslambda.streamifyResponse(async (event, responseStream, context) => {
responseStream = awslambda.HttpResponseStream.from(responseStream, {
statusCode: 200,
headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache" },
});
let messages;
try {
messages = JSON.parse(event.body ?? "{}").messages;
if (!Array.isArray(messages) || JSON.stringify(messages).length > MAX_PROMPT_CHARS) throw new Error();
} catch {
send(responseStream, { type: "error", code: "bad_request" });
responseStream.end();
return;
}
// Lambda keeps running after a client disconnect, so bound the work ourselves.
const abort = new AbortController();
const budgetMs = Math.max(1000, context.getRemainingTimeInMillis() - 3000);
const deadline = setTimeout(() => abort.abort(), budgetMs);
send(responseStream, { type: "session.start", schemaVersion: 1, requestId: context.awsRequestId });
try {
const response = await bedrock.send(
new ConverseStreamCommand({ modelId: MODEL_ID, messages, inferenceConfig: { maxTokens: MAX_TOKENS } }),
{ abortSignal: abort.signal }
);
for await (const ev of response.stream) {
if (ev.contentBlockDelta?.delta?.text) {
send(responseStream, { type: "delta", text: redact(ev.contentBlockDelta.delta.text) });
} else if (ev.messageStop) {
send(responseStream, { type: "stop", reason: ev.messageStop.stopReason });
} else if (ev.metadata?.usage) {
send(responseStream, { type: "usage", ...ev.metadata.usage });
}
}
send(responseStream, { type: "done" });
} catch (err) {
send(responseStream, { type: "error", code: abort.signal.aborted ? "deadline" : (err.name ?? "upstream") });
} finally {
clearTimeout(deadline);
responseStream.end();
}
});HttpResponseStream.from has sent the 200 status and headers, you can’t change them. Errors after that point have to travel in-band as an error event, which is why the client contract below includes one. On Node.js 24, Lambda no longer waits for unresolved promises after the handler returns or the stream ends, so await anything you start, such as metric flushes.
Step 3: Test it from a terminal
The AWS tutorial usescurl with SigV4 signing. --no-buffer makes curl print each chunk as it arrives, so you can see the stream working:
bash
You should see a curl --no-buffer --request POST "https://abcdefghijklm7nop7qrs740abcd.lambda-url.us-east-1.on.aws/" \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
--header "x-amz-security-token: $AWS_SESSION_TOKEN" \
--aws-sigv4 "aws:amz:us-east-1:lambda" \
--header "content-type: application/json" \
--data '{"messages":[{"role":"user","content":[{"text":"Explain this stack trace in 3 bullets."}]}]}'session.start line, then a series of delta lines, then stop, usage and done. If everything arrives in one burst at the end, jump to the troubleshooting table.
Step 4: Re-stream from your backend
Browsers can’t sign SigV4 requests safely, so most teams put a small backend in front. It authenticates the user, then callsInvokeWithResponseStream and forwards each payload chunk:
typescript
Notice that the payload wraps the request in a import { LambdaClient, InvokeWithResponseStreamCommand } from "@aws-sdk/client-lambda";
const lambda = new LambdaClient({});
export async function proxyAgentStream(res, payload) {
const out = await lambda.send(new InvokeWithResponseStreamCommand({
FunctionName: "coding-agent-stream",
Payload: Buffer.from(JSON.stringify({ body: JSON.stringify(payload) })),
}));
res.setHeader("Content-Type", "application/x-ndjson");
for await (const event of out.EventStream) {
if (event.PayloadChunk?.Payload) res.write(event.PayloadChunk.Payload);
if (event.InvokeComplete?.ErrorCode) {
res.write(JSON.stringify({ type: "error", code: event.InvokeComplete.ErrorCode }) + "\n");
}
}
res.end();
}body field. The handler reads event.body because function URLs deliver an HTTP-style event, so a direct SDK invoke needs to imitate that shape (or the handler needs to accept both).
A client contract your IDE plugins can rely on
Streams get consumed by more than one client: the web app, the IDE extension, the CLI. Write the event types down and version them, so you can add fields without breaking old clients:| Event | When | Fields |
|---|---|---|
session.start | First line, always | schemaVersion, requestId |
delta | Each text chunk from the model | text |
ping | While a tool runs and no tokens flow | ts |
stop / usage | Model finished | reason; token counts |
done / error | Last line, exactly one of them | code for errors |
Cost, cancellation and observability
Because a disconnected client doesn’t stop the function, your controls have to live inside the function:maxTokens on every request, the soft deadline above, and a function timeout that matches the longest answer you’re willing to pay for. For metrics, record time to first byte (TTFB) when the first delta is written, plus token usage from the usage event. A one-line Embedded Metric Format log turns that into a CloudWatch metric without an extra API call:
typescript
// Call once, when the first delta is written.
function recordTtfb(startedAt, stage) {
console.log(JSON.stringify({
_aws: {
Timestamp: Date.now(),
CloudWatchMetrics: [{ Namespace: "CodingAgent/Stream", Dimensions: [["Stage"]],
Metrics: [{ Name: "TimeToFirstByteMs", Unit: "Milliseconds" }] }],
},
Stage: stage,
TimeToFirstByteMs: Date.now() - startedAt,
}));
}Worked example: streaming a diff review
- The IDE extension sends a diff and the question “anything risky here?” to your backend, which checks the user’s session.
- The backend calls
InvokeWithResponseStream. The handler sendssession.startimmediately, so the extension can show “reviewing…” with a request ID. - Bedrock starts producing text. Each
deltaline is painted in the review panel as it arrives, so the engineer starts reading the first findings while the model is still writing. - The model finishes. The handler sends
stop(reasonend_turn), ausageline for billing, thendone. - On another day, a huge diff makes the review run long. Three seconds before the function timeout, the soft deadline aborts the Bedrock call, and the extension receives
{"type":"error","code":"deadline"}and suggests splitting the diff, instead of hanging.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Whole response arrives at once | URL still in BUFFERED mode, or you’re testing in the Lambda console | Set RESPONSE_STREAM; test with curl --no-buffer |
awslambda is not defined | Running outside the Lambda Node.js runtime (for example, locally) | Stub the global in local tests, or test against the deployed function |
| Python handler never streams | Python managed runtimes don’t support native streaming | Use Node.js, a custom runtime, or the Lambda Web Adapter |
| 403 from the function URL | Caller lacks lambda:InvokeFunctionUrl or lambda:InvokeFunction, or SigV4 signed for the wrong service | Grant both actions; sign for service lambda |
| Streaming fails for a function in a VPC | Function URLs don’t support response streaming within a VPC environment | Invoke with InvokeWithResponseStream through a Lambda interface endpoint |
| API Gateway still buffers | Integration transfer mode is BUFFERED, or it’s an HTTP API | Use a REST API with an AWS_PROXY integration and STREAM mode |
| Stream cuts off mid-answer | Function timeout reached, or an idle timeout upstream | Soft deadline before timeout; send ping during tool waits |
| Large downloads slow down after a while | Bandwidth is capped at 2 MBps after the first 6 MB | Expected; serve big files from S3 instead |
Production checklist
- ☐ Invoke mode
RESPONSE_STREAM(function URL) orSTREAM(API Gateway REST) - ☐
AWS_IAMauth; callers have both invoke permissions; CORS limited to your origins - ☐ Handler writes deltas immediately and always ends with
doneorerror - ☐ Model ID and
maxTokenscome from config, never from the request - ☐ Soft deadline a few seconds before the function timeout
- ☐ Client contract documented and versioned with
schemaVersion - ☐ TTFB and token usage emitted as metrics; errors alarmed
- ☐ Long tool runs moved off the stream to an async job
FAQ
Can API Gateway stream Lambda responses now?
Yes, REST APIs can. Set the response transfer mode of a Lambda proxy (AWS_PROXY) integration to STREAM. Streams can run for up to 15 minutes, and the first 10 MB aren’t bandwidth-limited. HTTP APIs aren’t covered by this feature.
Is streaming a replacement for WebSockets?
No. Streaming is one request with a long, incremental response. WebSockets fit when the server needs to push events after the request ends, or when the client sends messages mid-session, such as “stop” or a follow-up.Does streaming cost more?
You pay for function duration either way, and a stream runs as long as the model takes, just as a buffered call would. The difference is that the user sees output sooner. Check the Lambda and API Gateway pricing pages, since API Gateway charges for response streaming.How do tool calls fit into a stream?
Emit atool.start event, run the tool, send pings if it takes a while, emit tool.end, then start a new ConverseStream call with the tool result. Anything longer than a few minutes belongs in an async job.
Official documentation
- Response streaming for Lambda functions (limits, bandwidth, runtimes, VPC, billing on disconnect)
- Writing response streaming-enabled functions
- Invoking streaming functions with function URLs (invoke modes)
- Control access to Lambda function URLs (both invoke permissions)
- Stream the integration response in API Gateway
- ConverseStream API reference
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- Python String Methods: Every str Method With Real Production Examples
- API Debugging in 30 Seconds: The Fastest Ways to Fix API Issues
Newly added
- n8n vs AI Agent: I Gave Both the Same Job (Local, Ollama)
- Local AI Video Generator: Build Your Own App (Free, Python, No GPU)
- AI Wrote Raw Binary Instead of Code. Here’s What Happened (WebAssembly)
- I Built a Browser That Only Runs Encrypted Binary (Ep 1)
- RAG From Scratch: Chunking, Embeddings, Reranking and Evaluation (Real Numbers)
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.