Mutation-Check Your Generated Component Tests

Tier: Core · Workflow area: Testing & quality

Generated test suites fail in a specific way: they execute lots of code (great coverage) while asserting little of consequence (decorative assertions). StrykerJS makes that failure measurable, and turns "are these AI tests any good?" into a number you can gate on.

Prerequisites

• •
  A component test suite (Vitest or Jest + RTL) that is green and reasonably fast — mutation time scales with suite time.
• •
  StrykerJS installed with the matching runner: npm i -D @stryker-mutator/core @stryker-mutator/vitest-runner (or jest-runner).
• •
  An hour the first time; incremental runs after that are much cheaper.

The recipe

1. 1.

   Scope the first run tightly. Whole-repo mutation runs on day one produce a wall of noise and a coffee-length wait. Target the component you care about:

   Copy

   // stryker.config.json
   {
     "testRunner": "vitest",
     "mutate": ["src/components/PaymentForm.tsx"],
     "coverageAnalysis": "perTest",
     "incremental": true,
     "thresholds": { "high": 85, "low": 70, "break": 60 }
   }
   

   Copy

   npx stryker run
   

2. 2.

   Read the score as a verdict on assertions, not on effort. Sample output:

   Copy

   #  Mutant survived: src/components/PaymentForm.tsx:47
     -  if (attempts >= MAX_ATTEMPTS) lockForm();
     +  if (attempts > MAX_ATTEMPTS) lockForm();
   Ran 12 tests. 12 passed.
   Mutation score: 64.2% (killed 52, survived 24, timeout 3, no coverage 2)
   

   Every survivor is a concrete bug your suite would ship. A boundary flip surviving means no test exercises the boundary; a deleted onSubmit call surviving means no test asserts the callback fired. "No coverage" mutants are code the suite never runs at all — different problem, same report.

3. 3.

   Triage survivors into three buckets before fixing anything. Read each surviving mutant and classify: missing assertion (the behavior is exercised but never checked — the AI-suite classic), missing scenario (nothing tests that branch: boundaries, error paths, the unhappy roads generators skip), and equivalent mutant (the mutation cannot change observable behavior — e.g. mutating a value that is immediately overwritten; ignore these, and annotate if the tooling supports it). The bucket ratio is diagnostic: generated suites skew heavily toward missing assertions.

4. 4.

   Feed survivors back to the generator — this loop is the playbook. The mutant report is a perfect, concrete prompt:

   Copy

   These 8 mutants survived our test suite for PaymentForm.tsx (report
   attached: file:line, original code, mutated code). For each: state
   the user-visible behavior difference the mutation causes, then write
   the RTL test that kills it — accessible queries and userEvent only,
   per our test rules. If a mutation causes NO user-visible difference,
   say "equivalent" and do not write a decorative test for it.
   

   The "state the behavior difference first" clause matters: it stops the model from writing implementation-poking tests that technically kill the mutant while violating the behavior-first rules. Review the new tests with the same eye as any generated test — a bad test that kills a mutant is still a bad test.

5. 5.

   Re-run incrementally until the score plateaus honestly. With "incremental": true, re-runs only test what changed. Iterate steps 3–4 until survivors are all equivalent-or-accepted. Do not chase 100% — the last few points are usually equivalent mutants and defensive code; 85%+ on interactive components is a strong suite.

6. 6.

   Institutionalize the exam where it pays. Full-repo mutation in CI is too slow for most frontends, so be surgical: run Stryker (incremental) on changed components in nightly CI, and adopt a team rule that any newly generated suite ships with its mutation score in the PR description. The score replaces "the tests look thorough" in review conversations — a claim with a number beats an adjective.

Tool variations

• Vitest projects:@stryker-mutator/vitest-runner; keep the suite fast (mock network, fake timers) — mutation multiplies your suite time by hundreds of runs.
• Jest projects:@stryker-mutator/jest-runner with coverageAnalysis: "perTest" for the big speed win.
• Claude Code as the loop driver: it can run Stryker, parse the report, draft killer tests, and re-run — gate it on your review of each new test, and make it print the before/after score.
• Mobile: the same discipline exists as Pitest for Android/Kotlin (via gradle plugin) and Muter for Swift; scope to view-model and logic layers where suites are fast enough.

Quality check

• Score measured, not assumedthe generated suite has a mutation score on record before it is trusted as a refactor net.
• Every survivor triagedclassified missing-assertion / missing-scenario / equivalent, with equivalents annotated rather than gamed.
• Killer tests obey the house rulesnew tests are behavior-first with accessible queries, not mutant-specific implementation pokes.
• Score is in the PRnewly generated suites publish their number; reviewers see 85%, not "looks comprehensive."
• Runs stay affordableincremental mode on, scope surgical, so the exam actually keeps happening.