Adopt Self-Healing Locators with Eyes Open — and Catch the Wrong Heal

Tier: Core · Workflow area: Agentic test stack & self-healing

This playbook adopts self-healing deliberately: calibrate the healer against seeded changes before trusting it, scope where healing may act, and wire every heal into a review path so the wrong heal is caught before it lands.

Prerequisites

• •
  One healing mechanism chosen: Healenium (Selenium ecosystem, runs as a proxy with a locator DB), Cypress cy.prompt (AI-assisted commands), or Playwright's healer agent (proposes diffs post-failure).
• •
  Strong assertions in the pilot suite — verified, not assumed (mutation-test a sample if unsure).
• •
  A staging build you can deliberately mutate: one benign UI rename and one real behavior break.
• •
  A PR-based workflow for landing locator changes.

The workflow

1. 1.

   Pick a pilot slice, not the whole suite. Choose 20–40 stable, strongly-asserted tests on a UI area with real churn. Wire the healer there only — global adoption before calibration is how wrong heals reach main unnoticed.

2. 2.

   Set up the mechanism. Healenium example (Selenium/Java):

   Copy

   docker compose up -d   # healenium backend + selector DB
   

   Copy

   WebDriver delegate = new ChromeDriver();
   WebDriver driver = SelfHealingDriver.create(delegate);  // heals on NoSuchElement
   

   For Playwright, healing is not runtime magic — you run the healer agent on failures and receive diffs. That difference is a governance feature: keep it.

3. 3.

   Calibrate with a two-change drill before trusting anything. On the staging build, apply:

   • •
     Change A (benign): rename id="submit-btn" to id="checkout-submit" — same behavior.
   • •
     Change B (regression): the discount field no longer applies the discount, and its DOM moved.

   Run the pilot. The correct outcome: the healer heals A's locator and the test passes; B's test may find the moved element, but the assertion on the discounted total must still fail. If B goes green, you have witnessed the wrong heal — and learned your assertions or healing scope need work before rollout. Sample calibration record:

   Copy

   Change A submit-rename  : healed selector, test PASS            -> correct heal
   Change B discount-break : healed selector, total-assert FAIL    -> healing scoped OK
   Change B (weak variant) : healed selector, visibility-only PASS -> WRONG HEAL, blocked rollout
   

4. 4.

   Scope what healing may touch. Configure the healer to act on locator resolution only — never on assertions, waits, or expected values. Any tool or prompt-based healer that "fixes" an expected value has crossed from maintenance into evidence-tampering; that capability stays off.

5. 5.

   Route every heal to review. Healenium: enable the report/plugin so healed selectors surface per run, and export them; Playwright healer: heals are diffs by construction — open them as PRs. Either way, a heal is a proposed change with a required human decision. The companion audit-queue playbook defines the review protocol; the non-negotiable here is that no healed selector silently becomes the new baseline.

6. 6.

   Ask the one question per heal. For each healed locator: did the app's intended behavior change here? Selector drift → accept, commit the new locator (so the heal count returns to zero). Behavior change → reject, file the bug, and mark the heal as a caught wrong-heal in your log.

7. 7.

   Watch the heal rate as a health metric. A healthy pilot shows a burst after UI refactors and near-zero otherwise:

   Copy

   week 1: 14 heals (nav refactor)  | 14 accepted, 0 rejected
   week 4:  2 heals                 | 1 accepted, 1 REJECTED -> bug FE-2214 filed
   

   A sustained high rate means locator strategy is wrong (fix with role/label locators, ARIA snapshots), not that healing is working well.

Variations

• Cypress cy.prompt: intent-based steps re-resolve at runtime; because there is no diff artifact, compensate with strict assertions and a periodic re-recording review of what the prompt actually did.
• Playwright without a healer product: ARIA-snapshot assertions plus role-based locators eliminate most healing demand — often the better first move; adopt healing only for what remains.
• Vendor self-healing (mabl, testRigor, Applitools-class): same drill applies — run the two-change calibration in the POC and demand the wrong-heal rate in the scorecard before buying.
• Heal-in-CI-only: allow healing during nightly runs but fail PRs on unhealed selectors — keeps main honest while still harvesting healing's maintenance savings.

Quality check

• Calibration on record: the two-change drill ran before rollout, and the wrong-heal variant was demonstrably caught.
• Scope enforced: config shows healing limited to locator resolution; assertions and waits untouchable.
• Review trail: every heal has an accept/reject decision; accepted heals committed so the baseline stays versioned.
• Assertion strength maintained: periodic mutation-testing of the healed slice confirms tests still fail when behavior breaks.
• Metric watched: heal rate charted; spikes investigated as either refactor events or locator-strategy debt.

Pitfalls

• Healing a flaky suite. Async and data flakes get "healed" into different nondeterminism; diagnose flakes first (see the failure-signature playbook), heal locators second.
• Weak assertions under healing. The multiplier of doom — the healer finds an element, the visibility assert passes, the feature is dead. Assertion audit is a precondition, not a nicety.
• Silent baseline drift. Runtime healing without committing accepted selectors means the versioned suite no longer matches what runs. Export and commit accepted heals.
• Trusting confidence scores. Healenium-style similarity scores rank candidates; they say nothing about behavioral correctness. A 0.98-confidence heal can mask a bug perfectly.
• Global rollout on day one. Without calibration you have no idea of your wrong-heal exposure; the pilot exists to measure it.

Restricted-environment variant

Prompt-based healers (cy.prompt, LLM healer agents) send page structure to a model provider — on client engagements clear the tool and keep client data out of test environments it sees. Healenium's classical similarity engine runs fully self-hosted and usually clears strict policies; prefer it where model egress is blocked. If auditors require exact correspondence between versioned and executed tests, disable runtime healing entirely and use only diff-proposing healers with PR review — same benefit, cleaner trail.

Success criteria

• •
  Selector-maintenance time measurably down on the pilot slice, with the before/after numbers captured.
• •
  100% of heals reviewed; accepted heals committed; heal count returning to zero between refactors.
• •
  At least one wrong heal caught in calibration or production use — proof the safety net is real, not theoretical.
• •
  Assertion-strength spot checks (mutation runs) show no degradation since healing was enabled.