AI Test Generation: Property Tests That Catch Race Conditions

Unit tests that assert one happy path will not catch races in Node services. Prompt LLMs to emit property-based tests and concurrency harnesses that scramble interleavings, assert invariants, and quarantine flakes — not to rubber-stamp nondeterministic greens.

⚡ TL;DR: Feed the model the module + invariants; require fast-check properties and a deterministic concurrency harness (Atomics/worker_threads or controlled async queues); fail CI on flake quarantine growth; never accept retries: 3 as a fix. See event loop lag, Express production APIs, pytest fixtures for cross-language discipline.

Prompt for invariants, not examples

SYSTEM: You generate tests only. Output structured JSON { "properties": [...], "concurrency": [...] }.
Each property must name invariant, generator, and assertion.
No sleeps as synchronization. No retries to hide races.

// invariants for a ticket lock service
export const INVARIANTS = [
  "at most one holder for a ticketId at a time",
  "release without hold is a no-op error",
  "held count equals open acquires minus releases",
];

Property tests with fast-check

import fc from "fast-check";
import { TicketLock } from "../src/ticketLock";

test("exclusive hold invariant", async () => {
  await fc.assert(
    fc.asyncProperty(
      fc.array(fc.record({
        op: fc.constantFrom("acquire", "release"),
        id: fc.integer({ min: 1, max: 5 }),
      }), { minLength: 1, maxLength: 40 }),
      async (ops) => {
        const lock = new TicketLock();
        const held = new Map<number, number>();
        for (const op of ops) {
          if (op.op === "acquire") {
            const ok = await lock.tryAcquire(op.id);
            if (ok) held.set(op.id, (held.get(op.id) ?? 0) + 1);
            // ✅ invariant
            expect(held.get(op.id) ?? 0).toBeLessThanOrEqual(1);
          } else {
            await lock.release(op.id);
            held.set(op.id, Math.max(0, (held.get(op.id) ?? 0) - 1));
          }
        }
      }
    ),
    { numRuns: 100 }
  );
});

✅ Model fills generators + invariants you stated.
❌ Model writes one acquire then release example and calls it done.

Concurrency harness without flaky sleeps

import { Worker, isMainThread, parentPort, workerData } from "worker_threads";

// Controlled parallel acquires against shared resource mock
export async function storm(nWorkers: number, ticketId: number) {
  const workers = Array.from({ length: nWorkers }, () =>
    new Promise<boolean>((resolve, reject) => {
      const w = new Worker(__filename, { workerData: { ticketId } });
      w.on("message", resolve);
      w.on("error", reject);
    })
  );
  const results = await Promise.all(workers);
  // Exactly one true if lock is exclusive
  expect(results.filter(Boolean)).toHaveLength(1);
}

if (!isMainThread) {
  // worker body: attempt acquire once, post result
}

For event-loop races (shared in-memory maps, not worker_threads), inject a schedulable queue:

type Task = () => Promise<void>;
export class ManualScheduler {
  q: Task[] = [];
  enqueue(t: Task) { this.q.push(t); }
  async flushPermutations(limit = 50) {
    // illustrative: run a few shuffles — full permute explodes fast
    for (let i = 0; i < limit; i++) {
      const copy = [...this.q].sort(() => Math.random() - 0.5);
      for (const t of copy) await t();
    }
  }
}

Pair with event loop lag metrics so production races surface as lag, not only test failures.

Quarantine flakes — don’t bless them

// ci/flake-budget.ts
const MAX_QUARANTINED = 5;
// ❌ jest.retryTimes(3) on concurrency tests
// ✅ quarantine file with owner + issue link; CI fails if quarantine grows

LLM-generated tests that need retries are defective — regenerate with stronger synchronization or narrower scope.

Closing checklist

✅ Dos
– ✅ State invariants explicitly in the codegen prompt
– ✅ Prefer fast-check properties + controlled concurrency harnesses
– ✅ Assert exclusivity / conservation style invariants
– ✅ Cap flake quarantine with ownership
– ✅ Correlate with event-loop lag in prod

❌ Don’ts
– ❌ Don’t accept sleep(100) as synchronization
– ❌ Don’t hide races with retryTimes
– ❌ Don’t generate only example-based unit tests for locks/queues
– ❌ Don’t let agents skip failing concurrency tests
– ❌ Don’t ignore nondeterministic order dependencies in Express handlers — production API practices

Related reading

• Node.js Event Loop Lag: Catch p99 Stalls Before Users Feel Them
• Express.js Best Practices: Build Production-Ready APIs With Node.js
• Python Testing With pytest: From First Test to Advanced Fixtures and Mocking
• AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines

Last updated on September 11, 2026