Node Test Runner Shards: Parallel CI Without Introducing Flakes

node --test shards cut wall-clock CI in half — and will invent flakes if tests share disks, ports, clocks, or ordering assumptions. The senior setup isolates each shard as a process, partitions files deterministically, injects seeds for property tests, and serializes access to scarce resources (local DynamoDB, Redis, listen ports).

⚡ TL;DR: Shard by file list (--test-force-exit, separate processes), not by mutating global state inside one process; give each shard a unique PORT/TMPDIR; seed randomness; ban open-ended setTimeout asserts. Align with CI Failure Triage Bots so shard flakes are labeled separately from product regressions.

Deterministic file partitioning

// scripts/shard-tests.ts
import { createHash } from "node:crypto";
import { readdirSync } from "node:fs";
import { spawnSync } from "node:child_process";

const shardIndex = Number(process.env.SHARD_INDEX ?? 0);
const shardTotal = Number(process.env.SHARD_TOTAL ?? 1);

const files = readdirSync("test", { recursive: true })
  .map(String)
  .filter((f) => f.endsWith(".test.js") || f.endsWith(".test.ts"))
  .map((f) => `test/${f}`.replace(/\/+/g, "/"))
  .sort();

const mine = files.filter((f) => {
  const h = createHash("sha256").update(f).digest();
  return h.readUInt32BE(0) % shardTotal === shardIndex;
});

console.log(`shard ${shardIndex}/${shardTotal}: ${mine.length} files`);
const r = spawnSync(process.execPath, ["--test", "--test-reporter=spec", ...mine], {
  stdio: "inherit",
  env: {
    ...process.env,
    TMPDIR: `${process.env.RUNNER_TEMP}/t-${shardIndex}`,
    PORT: String(4100 + shardIndex),
    TEST_SEED: process.env.GITHUB_SHA ?? "local",
  },
});
process.exit(r.status ?? 1);

# .github/workflows/test.yml
strategy:
  fail-fast: false
  matrix:
    shard: [0, 1, 2, 3]
steps:
  - run: corepack enable && pnpm i --frozen-lockfile
  - run: pnpm build
  - name: Shard
    env:
      SHARD_INDEX: ${{ matrix.shard }}
      SHARD_TOTAL: 4
    run: pnpm tsx scripts/shard-tests.ts

Kill the usual flake sources

// test/helpers/ports.ts
import { createServer } from "node:net";

export async function listenEphemeral(): Promise<{ port: number; close: () => Promise<void> }> {
  const srv = createServer();
  await new Promise<void>((r) => srv.listen(0, "127.0.0.1", r));
  const addr = srv.address();
  if (!addr || typeof addr === "string") throw new Error("bad_addr");
  return {
    port: addr.port,
    close: () => new Promise((r, j) => srv.close((e) => (e ? j(e) : r()))),
  };
}

// ❌ Shared 3000 across shards
await app.listen(3000);

// ❌ Relying on test file order
assert.equal(globalCounter, 3);

Seeds for property and random tests

import { test } from "node:test";
import assert from "node:assert/strict";

const seed = process.env.TEST_SEED ?? "dev";

test("idempotent encode " + seed, () => {
  // derive PRNG from seed — failures must reprint seed
  assert.ok(seed.length > 0);
});

Process isolation beats in-process concurrency first

Prefer multiple node --test processes (shards) before enabling high in-process concurrency. When you do use concurrency, mark tests that touch FS/network as serialized.

import { describe, test } from "node:test";

describe("db integration", { concurrency: false }, () => {
  test("migrations", async () => { /* ... */ });
});

Closing checklist

✅ Dos
– ✅ Partition files with a stable hash, not directory size guesses
– ✅ Unique TMPDIR / ports per shard
– ✅ Print TEST_SEED on failure
– ✅ fail-fast: false so one shard doesn’t hide others
– ✅ Quarantine known flakes with owners and expiry

❌ Don’ts
– ❌ Don’t hardcode listen ports
– ❌ Don’t share mutable globals across files
– ❌ Don’t assert on timing with bare setTimeout
– ❌ Don’t run migrations from every shard against one DB
– ❌ Don’t merge shard JUnit without shard labels

Related reading

• CI Failure Triage Bots: Separate Flaky Noise from Real Regressions
• AI Test Generation: Property Tests That Catch Race Conditions
• Node.js Event Loop Lag: Catch P99 Stalls
• AST-Guided Edits: LLMs Propose Intents and Codemods Apply Them

Last updated on September 11, 2026