flaky test prevention Shipped
What this lens looks for
Flaky tests destroy confidence — quarantine immediately, fix or delete, never ignore. Review the test changes for every source of nondeterminism: tests MUST NOT share mutable state (each test arranges its own fixtures — flag class-level, module-level, or global mutable fields reused across tests); MUST NOT depend on execution order (no test relies on another having run first, no ordering-sensitive shared setup); unit tests MUST NOT make real network calls (require fakes/stubs/mocks at the boundary); MUST NOT use sleep() or timing-dependent assertions (require deterministic waits, callbacks, or condition polling instead of arbitrary delays); unit tests MUST NOT produce filesystem side effects (require temp directories with teardown cleanup). SHOULD-level: tests should not read the system clock directly — time should be injected as a dependency so date/time logic is deterministic. An intermittently-failing test is broken, not noise: it MUST be treated as a P1 bug — quarantined and then fixed or deleted, never retried into green or silently ignored.
What its verifier checks
No sleep() or timing/delay-based assertions in test bodies (deterministic waits, callbacks, or polling used instead); no real network/HTTP calls in unit tests (boundaries faked or stubbed); no shared class-level/module/global mutable state reused across tests (each test arranges its own fixtures); no execution-order dependency between tests; unit tests create no filesystem side effects outside temp directories and clean up in teardown; the system clock is injected rather than read directly; any test identified as intermittently failing is quarantined and fixed or deleted (treated as a P1 bug), not ignored or retried into passing.