Migrate a Legacy Selenium/Java Suite to Playwright/TypeScript with Parity Checks

Tier: Advanced · Workflow area: Automation engineering

This playbook migrates a legacy Selenium suite file-by-file with an AI coding agent, holding every converted spec to a parity gate: the new test must fail when the old one fails and catch the same seeded bugs, before the Java original is allowed to die.

Prerequisites

• •
  An agentic IDE assistant that can read both repos: Claude Code, Cursor, or Copilot in agent mode.
• •
  A working Playwright/TypeScript scaffold (npm init playwright@latest) with your team's config conventions committed first.
• •
  The legacy suite runnable locally or in CI — you need its pass/fail behavior as ground truth.
• •
  A seeded-bug branch of the app (or the ability to revert a known bug fix) for parity checks.

The workflow

1. 1.

   Inventory before you convert. Have the agent map the old suite, not translate it yet:

   Copy

   Read src/test/java/** in this repo. Produce a migration inventory table:
   test class | scenarios covered | page objects used | selectors (CSS/XPath/id counts) |
   waits (explicit/implicit/Thread.sleep) | test data source | flake markers (@Retry, ignored).
   Flag any test that is disabled, duplicated, or asserts nothing meaningful.
   Do not write any TypeScript yet.
   

   Kill the dead tests now. Migrating a disabled test is effort spent embalming a corpse.

2. 2.

   Convert one vertical slice first — page object plus its tests. Scope the prompt tightly:

   Copy

   Convert LoginPage.java and LoginTests.java to Playwright/TypeScript in tests/login.spec.ts.
   Rules:
   - Use role/label-based locators (getByRole, getByLabel); no XPath. If the old XPath
     encodes intent you cannot recover from the DOM, add a TODO with the original selector.
   - Replace every explicit wait and Thread.sleep with Playwright auto-waiting or
     expect(...).toPass — flag any sleep you cannot remove.
   - Preserve assertion INTENT exactly. Do not strengthen or weaken checks.
   - Match conventions in tests/example.spec.ts and playwright.config.ts.
   

3. 3.

   Run the parity gate on the slice. Parity has three legs:

   Copy

   # Leg 1: both green on healthy app
   mvn test -Dtest=LoginTests && npx playwright test tests/login.spec.ts
   
   # Leg 2: both red on the seeded-bug build
   git checkout seeded-bugs && mvn test -Dtest=LoginTests; npx playwright test tests/login.spec.ts
   

   Sample parity log entry you should be producing per file:

   Copy

   login.spec.ts  | old: 6 pass | new: 6 pass | seeded bug LGN-3: old FAIL / new FAIL  ✔ parity
   cart.spec.ts   | old: 9 pass | new: 9 pass | seeded bug CRT-1: old FAIL / new PASS ✘ ESCAPE
   

   Leg 3 is assertion review: diff old vs new assertions side by side. A converted test that passes on the broken build (an "escape", like cart.spec.ts above) means the agent dropped or weakened an assertion — the most common and most dangerous conversion defect.

4. 4.

   Feed escapes back, scoped. Paste the escaping test pair and the seeded bug description: "the Java test catches this bug, your TypeScript version does not — find the dropped assertion and restore it, change nothing else."

5. 5.

   Batch the long tail only after the pattern holds. Once 3–4 slices pass parity clean, let the agent convert in batches of 5–10 files per PR, each PR carrying its parity log. Never one 400-file PR — reviewability is the gate's whole value.

6. 6.

   Delete the Java original per file, only after parity. The old test is the oracle; deleting it early deletes your evidence. Track a burn-down: converted / parity-proven / deleted.

Variations

• Claude Code with both repos open: strongest option — it cross-references PageFactory fields against live DOM output and proposes role-based replacements with reasons.
• Cypress or WebdriverIO as target: identical workflow; only the locator-idiom rules in step 2 change.
• No seeded-bug build available: use mutation-style parity — temporarily break the feature by hand (wrong label, dead button) and confirm both suites fail. Weaker, but far better than green-on-green.
• Python/pytest-selenium source: same shape; the inventory step matters more because implicit waits hide in conftest fixtures.

Quality check

Before any batch PR merges:

• Parity log complete: every converted file shows old/new agreement on healthy and broken builds; zero unexplained escapes.
• Locator audit:grep -rn "xpath\|Thread.sleep\|waitForTimeout" tests/ returns only justified, commented survivors.
• Assertion diff reviewed: counts per test compared old vs new; any reduction explained in the PR description.
• Flake budget: run the new batch 5× (npx playwright test --repeat-each=5); anything under 100% goes back before merge.
• Runtime sanity: the Playwright batch should be dramatically faster; if it is not, hunt the leftover fixed waits.

Pitfalls

• Silently dropped assertions. The signature failure. Agents convert navigation faithfully and quietly lose the awkward final assert. Only the seeded-bug leg catches it reliably.
• XPath transliterated into CSS. Equally brittle, now in a new language. Demand role/label locators and make the TODO escape hatch visible.
• Sleeps reborn as waitForTimeout. A converted sleep is still a sleep. Grep for it; treat each as a defect.
• Page objects copied wholesale. Playwright fixtures often eliminate the class entirely; a faithful port of a bad abstraction is a missed exit. Let the agent propose the fixture rewrite in a separate, clearly-labeled commit.
• The 400-file heroic PR. Unreviewable, unbisectable, and it hides the one escape that matters. Batch small, log parity per file.

Restricted-environment variant

Client-owned suites at services companies: confirm the client's AI-tool policy before an agent reads their repo. If public assistants are barred, run a local coder model (Ollama, 14B+ coder class) for the conversion prompts — conversion is pattern-heavy and local models handle it acceptably — and keep the parity harness identical, since it is pure test execution. Strip client hostnames, credentials, and business-identifying test data from any prompt context; parity logs shared in reports should reference test IDs, not client URLs.

Success criteria

• •
  100% of live legacy tests converted, parity-proven, and the Java suite deleted — with a burn-down chart to show for it.
• •
  Zero escapes remaining: every seeded or hand-planted bug caught by both suites during migration.
• •
  No XPath, no sleeps, no waitForTimeout without a justifying comment in the new suite.
• •
  Suite runtime cut (typically 2–5×) and flake rate at or below the old suite's from day one.