Most browser/device matrices are archaeology — Safari 14 is still there because someone added it in 2021. This playbook rebuilds the matrix from your analytics, risk-ranks every cell, and gives a defensible answer to "why don't we test on X?" — AI does the aggregation; the traffic data holds the pen.
Tier: Foundation · Workflow area: Non-functional & visual testing
In about an hour you will turn a raw analytics export into a tiered compatibility matrix: which browser/OS/device combinations get full runs, which get smoke tests, and which are explicitly out of support — each decision traceable to a number.
Export usage the right shape. From GA4: Tech → Browser / OS / Device category, last 90 days, with sessions and a value metric (conversions or revenue). A configuration with 1.2% of sessions but 6% of checkout revenue is not a 1.2% priority. Export to CSV.
Have the assistant aggregate and cluster — arithmetic first, no opinions yet.
Here is a 90-day CSV of sessions by browser version, OS, and device
category, plus conversion counts. Tasks:
1. Collapse browser minor versions into current / current-1 / older.
2. Produce a table of configuration clusters with % sessions and
% conversions, sorted by sessions.
3. Compute cumulative coverage — show where we cross 80%, 95%, 99%.
Show your arithmetic. Do not recommend anything yet.
Chrome (current/-1) · desktop 41.2% sessions · 47.0% conv · cum 41%
Chrome/WebView · Android 22.8% · 17.1% · cum 64%
Safari (current/-1) · iOS 19.4% · 24.6% · cum 83%
Edge (current) · desktop 5.1% · 4.9% · cum 88%
Safari (current/-1) · macOS 4.4% · 4.2% · cum 93%
Firefox (current) · desktop 2.6% · 1.4% · cum 95%
Samsung Internet · Android 2.1% · 0.5% · cum 97%
"Show your arithmetic" is load-bearing: LLMs miscount, and percentage errors here misallocate your whole budget. Spot-check three rows against the raw export before proceeding.
Add the risk dimension from bug history. Paste the browser-tagged defect export: "Cross-reference: which clusters have disproportionate defect history relative to traffic share?" Safari's rendering quirks and Samsung Internet's WebView deviations typically punch above their traffic weight — a small-share, high-defect cell may deserve a tier bump.
Draft the tiered matrix with explicit rules.
Build a 3-tier matrix from the tables above:
- Tier 1 (full regression, every release): clusters until cumulative
sessions ≥ 85%, plus any cluster ≥ 3% of conversions.
- Tier 2 (smoke on release, full monthly): next clusters to 97%, plus
any high-defect-history cluster regardless of share.
- Tier 3 (unsupported — documented): everything else.
For every cell state the reason: the % figures or the defect pattern.
Flag any cluster where sessions and conversions rank differently.
Apply the human overrides — and write them down. Data cannot see the enterprise deal requiring Firefox ESR, the upcoming Android-first market launch, or the accessibility commitment that keeps an older Safari in scope. Add these as pinned rows labeled override: <reason>. An override with a written reason is strategy; an override without one is how the matrix rots back into folklore.
Ship it where decisions happen and set the review cadence. Land the matrix in the test-strategy repo, map Tier 1 into the BrowserStack/LambdaTest project config, and calendar a quarterly re-run of steps 1–4 (the prompt lives next to the matrix, so the refresh is a 20-minute job). Browser share moves; matrices must.
The matrix's authority comes from its traceability. When every cell answers "why?" with a number or a named override, the "can you just add Opera?" conversation takes thirty seconds — and ends with the data, not with whoever argued longest.
Publish a one-line support policy alongside the matrix — "we fully support browsers covering the last 85% of sessions, refreshed quarterly" — so future arguments are about the policy threshold, not about individual browsers. Policies scale; cell-by-cell debates do not.