Node Blob and File APIs: Efficient Multipart Upload Paths to S3

Node now speaks Blob / File the way browsers do — which tempts teams to await req.blob() and hold the entire upload in RAM before S3. That pattern OOMs API tasks on multi-hundred-MB media. The unfair advantage is a streaming path: Web streams or Node streams from the request into S3 multipart uploads, with part-size aligned to SDK defaults and checksums verified without a second full buffer.

⚡ TL;DR: Prefer @aws-sdk/lib-storage Upload with a stream body; avoid Buffer.concat / full arrayBuffer() on the API tier; use 8–16 MiB parts; abort multipart on client disconnect. Pair with Zero-Copy Node Streams and Lambda Ephemeral Storage when disk spill is required.

Stream request → S3 multipart

// src/upload-route.ts
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import type { IncomingMessage } from "node:http";
import { Readable } from "node:stream";

const s3 = new S3Client({});

export async function uploadFromRequest(req: IncomingMessage, key: string) {
  const contentType = req.headers["content-type"] ?? "application/octet-stream";
  const upload = new Upload({
    client: s3,
    params: {
      Bucket: process.env.UPLOAD_BUCKET!,
      Key: key,
      Body: Readable.fromWeb(req as any), // or pass Node Readable req directly
      ContentType: contentType,
    },
    queueSize: 4,
    partSize: 8 * 1024 * 1024, // ✅ 8 MiB parts
    leavePartsOnError: false,
  });

  req.on("aborted", () => {
    void upload.abort(); // ✅ don't leak incomplete multiparts
  });

  const result = await upload.done();
  return { etag: result.ETag, key };
}

// ❌ Double-buffer the world
const buf = Buffer.concat(await collect(req)); // OOM
await s3.putObject({ Bucket, Key, Body: buf });

When Blob/File help (and when they don’t)

// Worker / edge adapter receiving a Web File
export async function uploadFile(file: File, key: string) {
  // ✅ file.stream() is a ReadableStream — don't arrayBuffer() first
  const upload = new Upload({
    client: s3,
    params: {
      Bucket: process.env.UPLOAD_BUCKET!,
      Key: key,
      Body: Readable.fromWeb(file.stream() as any),
      ContentType: file.type || "application/octet-stream",
    },
    partSize: 16 * 1024 * 1024,
  });
  return upload.done();
}

Checksums and abort hygiene

// Lifecycle rule: abort incomplete multipart after 7 days
// Plus metrics on AbortMultipartUpload count
upload.on("httpUploadProgress", (p) => {
  metrics.gauge("upload.bytes", p.loaded ?? 0);
});

Closing checklist

✅ Dos
– ✅ Stream into @aws-sdk/lib-storage Upload
– ✅ Abort multipart on client disconnect
– ✅ Size parts 8–16 MiB; bound queueSize
– ✅ Prefer file.stream() over arrayBuffer()
– ✅ Lifecycle-abort incomplete multiparts in the bucket

❌ Don’ts
– ❌ Don’t Buffer.concat multi-hundred-MB uploads on API hosts
– ❌ Don’t leave abandoned multipart uploads forever
– ❌ Don’t use single PutObject for multi-GB objects
– ❌ Don’t ignore Content-Length / max body limits at the proxy

Related reading

• Zero-Copy Node Streams: Pipe Large S3 Objects Without Buffering
• Lambda Ephemeral Storage: Safe /tmp Strategies for Large Artifacts
• Brotli vs Gzip in Node: CPU Tradeoffs at the CDN Edge
• Node Fetch vs Got vs Undici: Throughput Under Real TLS Churn

Last updated on September 11, 2026