9 views
This guide focuses on Lambda Ephemeral Storage for production systems, with practical trade-offs for reliability, security, and cost.
/tmp is not a scratchpad you can ignore. Between 512 MB and 10,240 MB of ephemeral storage survives across warm invokes on the same execution environment — which is both a performance feature and a contamination footgun when unzip-and-process pipelines leave residue.
⚡ TL;DR: Size ephemeral storage from peak unzip + output + safety margin; namespace every invoke under/tmp/<requestId>; delete trees infinally; never assume empty/tmpon entry; monitortmpusage. Pair with Lambda Container Images vs Zip and Lambda Power Tuning.
Lambda Ephemeral Storage: production guidance
Zip bombs and nested archives expand far beyond download size. Budget:text
need ≈ download + uncompressed + output + 20% margin
typescript
// cdk
const fn = new lambda.Function(this, "ArtifactWorker", {
runtime: lambda.Runtime.NODEJS_20_X,
memorySize: 2048,
ephemeralStorageSize: Size.mebibytes(4096), // ✅ not default 512 for large zips
timeout: Duration.minutes(5),
});
Namespace and scrub
typescript
// tmpWorkspace.ts
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { randomUUID } from "node:crypto";
export async function withTmpWorkspace<T>(fn: (dir: string) => Promise<T>): Promise<T> {
const dir = path.join(os.tmpdir(), "inv", process.env.AWS_REQUEST_ID ?? randomUUID());
await fs.mkdir(dir, { recursive: true });
try {
return await fn(dir);
} finally {
// ✅ Always scrub — warm containers reuse /tmp
await fs.rm(dir, { recursive: true, force: true });
}
}
export async function assertFreshTmp() {
// Optional hygiene: remove orphaned inv/* older than N minutes
const root = path.join(os.tmpdir(), "inv");
try {
for (const name of await fs.readdir(root)) {
const p = path.join(root, name);
const st = await fs.stat(p);
if (Date.now() - st.mtimeMs > 15 * 60_000) {
await fs.rm(p, { recursive: true, force: true });
}
}
} catch {
/* ignore */
}
}
typescript
// ❌ Wrong: fixed path shared across warm invokes
const ZIP = "/tmp/input.zip";
await downloadTo(ZIP);
await unzip(ZIP, "/tmp/out"); // leaks + races under concurrency concurrency=1 still races across invokes
Contamination bugs to hunt
| Bug | Symptom | Fix |
|---|---|---|
| Leftover files | Wrong artifact processed | per-invoke dir + finally rm |
| Disk full | ENOSPC on warm path |
larger ephemeral + scrub |
| Symlink escape | Write outside workspace | refuse symlinks on extract |
| Parallel unzip to same folder | Corrupt trees | unique dirs only |
typescript
import { execFile } from "node:child_process";
import { promisify } from "node:util";
const execFileAsync = promisify(execFile);
export async function safeUnzip(zipPath: string, dest: string) {
// ✅ Example: unzip with -j avoided; use a library that blocks path traversal
await execFileAsync("unzip", ["-qq", "-o", zipPath, "-d", dest], { timeout: 120_000 });
// Post-pass: reject any extracted path that escapes dest via ..
}
Closing checklist
✅ Dos– ✅ Calculate peak uncompressed size before setting ephemeral storage
– ✅ Per-request directories under
/tmp– ✅
finally cleanup every path– ✅ Block zip-slip / symlink escapes
– ✅ Alarm on
ENOSPC / failed invokes
❌ Don’ts– ❌ Don’t reuse fixed
/tmp/input.zip paths– ❌ Don’t assume cold start wiped disk
– ❌ Don’t unzip untrusted archives without limits
– ❌ Don’t set 10 GB ephemeral “just in case” without cost review
Related reading
- Lambda Container Images vs Zip Deployment
- Lambda Power Tuning Memory Optimization
- AWS Lambda Best Practices
- Lambda Response Streaming Backpressure
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
