Lambda Ephemeral Storage: Safe /tmp Strategies for Large Artifacts

Lambda Ephemeral Storage: Safe /tmp Strategies for Large Artifacts

/tmp is not a scratchpad you can ignore. Between 512 MB and 10,240 MB of ephemeral storage survives across warm invokes on the same execution environment — which is both a performance feature and a contamination footgun when unzip-and-process pipelines leave residue.

⚡ TL;DR: Size ephemeral storage from peak unzip + output + safety margin; namespace every invoke under /tmp/<requestId>; delete trees in finally; never assume empty /tmp on entry; monitor tmp usage. Pair with Lambda Container Images vs Zip and Lambda Power Tuning.

Size for unzip peaks

Zip bombs and nested archives expand far beyond download size. Budget:

need ≈ download + uncompressed + output + 20% margin
// cdk
const fn = new lambda.Function(this, "ArtifactWorker", {
  runtime: lambda.Runtime.NODEJS_20_X,
  memorySize: 2048,
  ephemeralStorageSize: Size.mebibytes(4096), // ✅ not default 512 for large zips
  timeout: Duration.minutes(5),
});

Namespace and scrub

// tmpWorkspace.ts
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { randomUUID } from "node:crypto";

export async function withTmpWorkspace<T>(fn: (dir: string) => Promise<T>): Promise<T> {
  const dir = path.join(os.tmpdir(), "inv", process.env.AWS_REQUEST_ID ?? randomUUID());
  await fs.mkdir(dir, { recursive: true });
  try {
    return await fn(dir);
  } finally {
    // ✅ Always scrub — warm containers reuse /tmp
    await fs.rm(dir, { recursive: true, force: true });
  }
}

export async function assertFreshTmp() {
  // Optional hygiene: remove orphaned inv/* older than N minutes
  const root = path.join(os.tmpdir(), "inv");
  try {
    for (const name of await fs.readdir(root)) {
      const p = path.join(root, name);
      const st = await fs.stat(p);
      if (Date.now() - st.mtimeMs > 15 * 60_000) {
        await fs.rm(p, { recursive: true, force: true });
      }
    }
  } catch {
    /* ignore */
  }
}
// ❌ Wrong: fixed path shared across warm invokes
const ZIP = "/tmp/input.zip";
await downloadTo(ZIP);
await unzip(ZIP, "/tmp/out"); // leaks + races under concurrency concurrency=1 still races across invokes

Contamination bugs to hunt

Bug Symptom Fix
Leftover files Wrong artifact processed per-invoke dir + finally rm
Disk full ENOSPC on warm path larger ephemeral + scrub
Symlink escape Write outside workspace refuse symlinks on extract
Parallel unzip to same folder Corrupt trees unique dirs only
import { execFile } from "node:child_process";
import { promisify } from "node:util";
const execFileAsync = promisify(execFile);

export async function safeUnzip(zipPath: string, dest: string) {
  // ✅ Example: unzip with -j avoided; use a library that blocks path traversal
  await execFileAsync("unzip", ["-qq", "-o", zipPath, "-d", dest], { timeout: 120_000 });
  // Post-pass: reject any extracted path that escapes dest via ..
}

Closing checklist

✅ Dos
– ✅ Calculate peak uncompressed size before setting ephemeral storage
– ✅ Per-request directories under /tmp
– ✅ finally cleanup every path
– ✅ Block zip-slip / symlink escapes
– ✅ Alarm on ENOSPC / failed invokes

❌ Don’ts
– ❌ Don’t reuse fixed /tmp/input.zip paths
– ❌ Don’t assume cold start wiped disk
– ❌ Don’t unzip untrusted archives without limits
– ❌ Don’t set 10 GB ephemeral “just in case” without cost review

Related reading

Last updated on September 11, 2026


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply