BXUP Track progress & test yourself — free
QA & Testing · Playbook · Foundation · ~6 min read
QA & TESTING · PLAYBOOK · FOUNDATION

Turn Messy Repro Evidence into Filed-Quality Defect Reports

The gap between "found a bug" and "filed a report a developer can act on" is thirty minutes of tedium — transcribing steps from a recording, digging the failing request out of a HAR file, formatting environment details. AI with vision closes that gap in minutes; you keep the two steps that make a report trustworthy: verifying the repro and checking for duplicates.

Tier: Foundation · Workflow area: Triage, flakes & defect intelligence

This playbook converts raw evidence — screen recordings, screenshots, console logs, HAR files — into a complete, severity-argued defect report, with a mandatory personal repro before filing and a semantic duplicate check against the existing backlog.

USE IT WHEN

Evidence is rich and time is short

  • You have a recording or screenshot set plus logs, and writing it up properly keeps sliding to "later".
  • Reports from your team bounce back with "cannot reproduce" or "need more info" — the template discipline below fixes that.
  • Exploratory sessions (including Playwright MCP sessions) produce finding batches that all need filing at once.
SKIP IT WHEN

The shortcut is the risk

  • You have not seen the bug yourself and cannot verify it — never file a report you have not reproduced or personally witnessed; AI-drafted fiction with your name on it burns trust fast.
  • The evidence contains customer PII you cannot mask — clean it first (masking playbook) or draft by hand.
  • It is a one-line typo report — just file it; drafting ceremony costs more than it saves.

Prerequisites

  • A vision-capable assistant (Claude with image/video-frame input, or ChatGPT equivalent).
  • The raw evidence gathered per bug: recording or screenshots, console log, HAR export (browser DevTools → Network → save HAR), environment facts.
  • Your tracker's field conventions (Jira/Linear template, severity definitions).
  • Embedding-capable search over existing issues, or at minimum tracker keyword search, for the dedupe step.

The workflow

  1. Capture once, capture everything. At the moment of discovery: save the recording or grab the screenshot sequence, export the console log, save the HAR, note build/branch, browser/OS, account, and time (for server-log correlation). Thirty seconds now saves the "what build was that?" round-trip later.

  2. Draft with a structure-forcing prompt. Attach the evidence and pin the format:

    Attached: 6 screenshots from a screen recording, console log, HAR file.
    Draft a defect report with EXACTLY these sections:
    TITLE (symptom + location, under 12 words)
    STEPS TO REPRODUCE (numbered, from the screenshots — only what is visible)
    EXPECTED / ACTUAL (one line each)
    EVIDENCE (quote the exact console errors and the failing HAR request:
      method, URL, status, response body excerpt)
    ENVIRONMENT (from what I list below)
    SEVERITY-SUGGESTION with a one-sentence argument
    Rules: do not invent steps you cannot see; mark gaps as [UNCLEAR — verify].
    Environment: build 2.14.0-rc3, Chrome 126/macOS, staging, user role: buyer.
    

    The [UNCLEAR — verify] instruction is the honesty valve — it converts hallucination pressure into flagged gaps you resolve in step 3. Sample of the useful part of a draft:

    EVIDENCE
    Console: "TypeError: Cannot read properties of undefined (reading 'variantId')
              at CartItem.tsx:87" (screenshot 4, timestamp 00:41)
    HAR: POST /api/cart/items -> 500, response: {"error":"variant_id required"}
         preceding GET /api/products/8812 returned variants: [] (empty array)
    STEP 3: [UNCLEAR — verify] whether the size selector was left unset or reset itself
    

    Notice what the model did with the HAR: it found the failing request and the preceding response that explains it — that correlation is the highest-value thing vision-plus-logs drafting buys you.

  3. Reproduce it yourself — the non-negotiable gate. Follow the drafted steps literally on a clean session. Fix any step the draft got wrong, resolve every [UNCLEAR], and trim to the minimal repro (cut steps until the bug stops, restore the last one). If you cannot reproduce, the report does not get filed — it goes back to evidence-gathering, and the draft's job was to reveal that early, not to paper over it.

  4. Dedupe semantically before filing. Same bug, different words is the classic tracker rot. Search the backlog for symptom, component, and error-string matches — or better, embed the draft against open issues:

    Here is my draft report and the 12 closest existing issues (titles + descriptions).
    For each: DUPLICATE / RELATED / DISTINCT with one line of reasoning.
    A duplicate must share root symptom and trigger, not just the same page.
    

    Exact duplicate → add your evidence as a comment on the original. Related → file yours and link. The model proposes; you decide — near-duplicates with different root causes (same error, different endpoint) deserve separate reports.

  5. File with severity argued, not asserted. Keep the model's severity as a suggestion; you own the call, using your team's definitions (user impact × frequency × workaround). Attach the recording, log, HAR, and — if you ran one — the Playwright trace. A report a developer can start fixing without a single follow-up question is the bar.

  6. Batch mode for session output. After an exploratory session with 5+ verified findings, run steps 2–5 per finding in one conversation — the shared session context keeps environment details consistent — but never let batching erode the per-bug repro gate.

Key Takeaway:

AI turns evidence into prose; only reproduction turns prose into a report. The drafted steps are a hypothesis about what you saw — walking them yourself on a clean session is the verification, and the [UNCLEAR] flags are where the model is telling you exactly where to look.

Variations

  • Video-native drafting: where the assistant accepts video, feed the recording directly; otherwise extract frames at the decision points (ffmpeg -i bug.mov -vf fps=1 frames/f%03d.png) and attach the sequence.
  • Trace-first for automated finds: when a Playwright run caught the bug, the trace viewer's timeline plus the failure screenshot replaces most manual evidence — prompt from the trace's action list.
  • API-only bugs: skip vision; the HAR (or curl transcript) plus expected-contract excerpt from the OpenAPI spec makes a tighter report than any screenshot.
  • Tracker-integrated: with a Jira/Linear MCP connection, the assistant can file the approved draft directly — keep filing behind your explicit "file it" instruction, never automatic.

Quality check

  • Repro verified: you personally walked the final steps on a clean session; minimal-step count noted.
  • No invented content: every step traceable to evidence; all [UNCLEAR] flags resolved before filing.
  • Evidence quoted, not gestured: exact error strings, exact failing request with status and body excerpt; attachments included.
  • Dedupe decision recorded: duplicate/related/distinct verdicts kept (a one-line note in the report suffices).
  • Zero follow-up questions: track whether developers bounce your reports — the metric that proves the playbook.

Pitfalls

  • Filing the unreproduced draft. The failure that poisons everything downstream: a fluent, specific, wrong report. The repro gate exists because AI drafts read verified even when they are not.
  • Hallucinated steps between screenshots. The model interpolates what "must have" happened between frame 3 and 4. The rules line ("only what is visible") plus your walkthrough catch it.
  • PII in attachments. HAR files carry auth tokens and cookies; recordings show real customer data. Scrub HARs (delete Authorization headers, cookie values) and mask data before anything reaches an external model or a public tracker.
  • Severity inflation by fluency. Confident prose makes everything sound critical. Severity comes from your definitions, not the draft's adjectives.
  • Duplicate-blindness on batch filings. Ten findings filed fast can include two reports of one root cause; run the dedupe step within your own batch too.

Restricted-environment variant

Client engagements: repro evidence is client data by definition. Use only client-approved assistants; scrub HARs and logs with the masking pipeline before drafting; keep recordings inside the client's storage. If no assistant is approved for their data, invert the flow — write the evidence sections yourself and use AI only to polish structure and wording from an already-scrubbed skeleton. File in the client's tracker under their conventions, and disclose AI-assisted drafting where their delivery standards require it.

Success criteria

  • Report writing time drops from ~30 minutes to under 10 per bug, including the repro walkthrough.
  • "Cannot reproduce" and "need more info" bounce-backs approach zero over a month of filings.
  • Duplicate filings caught before submission, with dedupe verdicts on record.
  • Developers start fixing from your reports without pinging you — the silent compliment that proves the quality.
Pro Tip

Save your five best filed reports as few-shot examples in the team prompt library. Pasting one exemplary report into the drafting prompt ("match this quality and format") lifts every future draft — your best filing becomes the template the whole team's AI drafts inherit.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in