AI collapses the two walls of performance testing — writing load scripts and reading latency distributions — but it also produces confident bottleneck theories from thin air. The rule of this playbook: AI drafts scripts and hypotheses; APM data and re-runs decide what goes in the report.
Tier: Core · Workflow area: Non-functional & visual testing
You will turn a described user journey into a runnable k6 (or JMeter) scenario, run it against a controlled environment, and use an assistant to interpret the percentile data — with every hypothesis verified against Grafana/APM evidence before anyone reports it.
k6 version) or JMeter 5.6+; a dedicated, prod-like target environment with test data.Prompt for the script from the journey, with realism constraints.
Write a k6 script for this journey against https://staging.example.com:
1. POST /api/login (test users user001–user500, pool them)
2. GET /api/catalog?page=1
3. POST /api/cart (1–3 random items)
4. POST /api/checkout — only 30% of iterations reach checkout
Requirements:
- Stages: ramp to 200 VUs over 5m, hold 10m, ramp down 2m.
- Think time: random 2–8s between steps (real users read pages).
- Thresholds: p95<400ms and error rate<1% per endpoint via tags.
- check() every response status AND a body field — a 200 error page
must not count as success.
- No hardcoded tokens; read credentials from env vars.
The think-time and checkout-ratio lines are what separate a load model from a DoS of your own staging.
Review the draft for the three classic AI script bugs: shared mutable state across VUs (login token reuse), missing sleep() so the model applies 10× intended load, and checks that only assert status 200. Then smoke-run it small:
k6 run --vus 5 --duration 60s checkout.js
Run the real profile and capture the summary.
http_req_duration..: avg=212ms min=45ms med=180ms max=8.2s p(90)=310ms p(95)=390ms
http_req_failed....: 0.82% ✓ 164 / ✗ 19,836
checks.............: 99.1%
Export per-endpoint data too (--summary-export or the JSON output) — aggregate numbers hide everything interesting.
Ask for interpretation — hypotheses only, explicitly framed.
Here are k6 results per endpoint and stage [paste], plus our target
(p95<400ms @ 200 VU). Interpret:
- Where does the distribution suggest saturation vs steady degradation?
- avg is 212ms but max is 8.2s — what tail behaviors could explain this?
- List your top 3 bottleneck HYPOTHESES, each with the specific APM
evidence that would confirm or kill it. Label everything as
hypothesis — do not state causes as facts.
Good output looks like: "H1: DB connection-pool exhaustion during ramp — confirm via pool-in-use metric flatlining at max while wait time climbs; kill if pool never exceeds 60%."
Verify against APM before believing anything. Open Grafana for the run window and check each hypothesis's named evidence. Typical outcome: one confirmed (pool maxed at 100/100 from minute 6), one killed (CPU never passed 40%), one inconclusive (needs a re-run with query logging). Only confirmed items — with the graph attached — appear in the report.
Re-run to prove any fix and to check repeatability. One run is an anecdote. Same profile, post-fix: p95 moved 390→240 ms and the 8-second tail vanished, or it didn't. Two runs with wildly different results means an uncontrolled environment — say that in the report rather than averaging the fiction.
The most damaging output of AI-assisted performance work is a plausible, unverified bottleneck story in a stakeholder deck. "The AI suggested it was the connection pool" is not a finding. The graph of the pool at 100/100 is.
Honest interpretation is a chain of custody: distribution → hypothesis → named APM evidence → re-run. AI accelerates every link but is allowed to own only the middle two — the graphs and the re-runs belong to reality.