Test i18n with Pseudo-Localization and Vision-Model Layout Checks

Tier: Advanced · Workflow area: Non-functional & visual testing

This playbook builds a pseudo-loc pipeline into your build, screenshots the critical screens across locales, and uses a vision model as a tireless first-pass reviewer with a human escalation path for everything judgment-shaped.

When to use

Prerequisites

• •
  Externalized strings (i18next, ICU MessageFormat, gettext — any resource-file scheme).
• •
  A pseudo-loc transform: pseudo-localization (npm), i18next-pseudo, or a 20-line script the assistant writes against your resource format.
• •
  Playwright (or your E2E runner) able to launch the app with a locale switch, and a vision-capable model (Claude or GPT) for screenshot review.

The recipe

1. 1.

   Generate the pseudo-locale as a build artifact. Transform your base strings into an accented, expanded, bracketed variant:

   Copy

   npx pseudo-localization ./locales/en.json > ./locales/en-XA.json
   

   Copy

   { "checkout.submit": "[Þŀàçè ýöûŕ öŕðèŕ ñöŵ one two]" }
   

   The three tricks each catch a bug class: accents catch encoding issues, ~40% expansion catches truncation, brackets catch clipped ends. Any plain-English text visible in the pseudo-locale build is a hardcoded string — file it immediately.

2. 2.

   Screenshot the critical screens per locale variant.

   Copy

   for (const locale of ['en-XA', 'ar', 'de', 'ja']) {
     const page = await browser.newPage({ locale });
     await page.goto(`/checkout?lng=${locale}`);
     await page.screenshot({ path: `shots/${locale}/checkout.png`, fullPage: true });
   }
   

   Include RTL even before Arabic translations exist — pseudo-RTL (en-XB style, direction flipped) finds mirroring bugs early. Keep the set to your 15–25 critical screens; this is a sweep, not an archive.

3. 3.

   Vision-model first pass, tightly scoped to mechanics.

   Copy

   You are reviewing UI screenshots for LAYOUT defects only. For each image:
   - Truncated text (ellipsis or clipped glyphs) — quote the visible fragment.
   - Text overflowing its container or overlapping other elements.
   - Buttons/inputs whose label no longer fits.
   - RTL screens: elements that failed to mirror (icons, chevrons, progress).
   - Text rendered as tofu boxes (missing glyphs).
   Report as a table: screen, element, defect type, confidence (high/med/low).
   Do NOT comment on translation quality, wording, or aesthetics.
   

   Sample output you should expect:

   Copy

   | checkout | "Place your ord…" submit btn | truncation    | high |
   | profile  | date joined overlaps avatar  | overlap       | med  |
   | cart-ar  | back-chevron points left     | RTL unmirrored| high |
   

4. 4.

   Human-verify the high/medium findings. Open each flagged screenshot; confirm or reject. Vision models are strong on truncation and overlap, weaker on subtle spacing — expect a real-but-manageable false-positive rate, and track it (step 6 uses it).

5. 5.

   Escalate the judgment questions to native speakers — with a curated packet. Everything the model was told not to judge still needs judging: idioms, formality level, culturally loaded imagery, date/name conventions. The AI's legitimate role here is triage: "From these strings, list ones with idioms, humor, or cultural references likely to need reviewer attention in Japanese." Send native reviewers that shortlist plus screenshots-in-context instead of a 4,000-row spreadsheet — reviewer hours are your scarcest resource; spend them where machine confidence is lowest.

6. 6.

   Automate the sweep per release and calibrate. Wire steps 1–3 into CI on release branches. Each cycle, record the model's precision on verified findings. If high-confidence findings are >90% real, let medium-confidence ones batch weekly instead of blocking; if precision drops, tighten the prompt or shrink its scope. This is calibrated trust applied to vision.

Variations

• No E2E harness yet: run steps 1 and 3 on manually captured screenshots — the pipeline degrades gracefully to a checklist-driven manual sweep with an AI reviewer.
• Storybook shops: pseudo-localize at the component level and screenshot stories; catches fixed-width components before any page assembles them.
• Real-translation regression: once actual locales exist, rerun steps 2–4 on every translation drop; new strings are the highest-risk strings.

Quality check

• •
  Pseudo-locale build renders zero plain-English strings on critical screens.
• •
  Every high-confidence vision finding human-verified within the release cycle; precision tracked.
• •
  RTL screens reviewed with direction actually flipped, not just translated.
• •
  Native-speaker sign-off recorded per locale for the judgment shortlist; legal copy verified by a human in every shipped locale.

Pitfalls

• Pseudo-loc leaking to users. Gate en-XA behind build flags; it has reached production at real companies and it looks exactly as broken as you imagine.
• Concatenated strings that survive pseudo-loc."You have " + n + " items" may render fine in en-XA and still be untranslatable (word order). Pair this pipeline with a lint for string concatenation in UI code.
• Vision-model overreach. Given loose prompts, models happily critique translations they cannot actually evaluate. The "layout only" scope line is a hard boundary; re-add it every session.
• Screenshot rot. Sweeping all screens every build drowns signal. Critical screens per release; full sweep per locale launch.