Design-Doc Adversarial Review and Trade-off Steelmanning Before Humans See It

Tier: Core · Workflow area: Architecture & Design

AI will not design your system — but a reasoning-capable model is a tireless, unembarrassable adversary: it will attack your draft from four directions and argue both sides of your trade-offs with a straight face. The rules that make this work: you draft first (the design stays yours), the attack is structured (personas, not "any thoughts?"), and every claim the model makes gets verified before it changes your design.

When to use

• •
  You have a v1 design doc for something that matters — new service, rate limiter, data model, sync-vs-async boundary — and human review is scheduled.
• •
  A build-vs-buy / SQL-vs-NoSQL / queue-choice decision where the team is anchored on one side and needs the other side argued at full strength.
• •
  Pre-mortem duty: "how does this fail at 10x load / during a region outage / when the intern misconfigures it?"

Skip when you have no draft — asking AI to attack a vague idea produces vague attacks. And never substitute this for human review; it is the filter before it, so humans spend their attention on judgment, not on catching missing timeout handling.

Prerequisites

• •
  A written v1 design: problem, constraints (scale numbers, latency budget, team size, budget), proposed approach, explicitly listed assumptions. Attacks are only as sharp as the constraints are concrete.
• •
  A reasoning-strong model (Claude with extended thinking, o-series, Gemini) — this workflow is one of the clearest cases where model depth visibly changes output quality.
• •
  A change log discipline: you will produce v2, and the diff is the deliverable.

The workflow

1. 1.

   Run the four-lens attack. One lens per prompt (batched lenses blur):

   Copy

   You are reviewing the attached design for an org-wide rate-limiter service.
   Lens 1 — FAILURE MODES. Attack only reliability: enumerate the failure modes
   this design does not handle. For each: trigger scenario, blast radius, and
   the specific section of the doc that is silent about it. Rank by
   likelihood x impact. Do not suggest fixes yet — attack only.
   

   Repeat for Lens 2 — scale cliffs (which numbers break first at 10x/100x, hot keys, thundering herds), Lens 3 — security (abuse, authz gaps, multi-tenant leakage; STRIDE-lite, or hand off to pb-37 for a full pass), Lens 4 — operability (how is it deployed, observed, debugged at 3 a.m., rolled back?).

2. 2.

   Triage the attack transcript. Sort every finding: (a) real gap → fix the design, (b) already handled → the doc failed to say so, add a sentence, (c) out of scope → add to an explicit non-goals section, (d) model misunderstanding → often means the doc is ambiguous; clarify anyway. Category (b) and (d) findings improve the document; (a) improves the system. Both matter in review.

3. 3.

   Steelman the trade-offs you already made. For each contested decision:

   Copy

   Our draft chooses a centralized Redis-based limiter over a client-side
   library. Argue the CLIENT-SIDE position at full strength — the case its
   best advocate would make, using our stated constraints (40 services,
   3-person platform team, p99 budget 5ms). Then argue the counter. Then list
   the 3 assumptions the decision actually hinges on, phrased as verifiable
   claims (e.g. "Redis round-trip inside the VPC stays under 1ms at p99").
   

4. 4.

   Verify the hinge assumptions — outside the chat. The model's "Redis adds ~0.5ms" is a plausibility, not a measurement. Benchmark it, check the vendor limits page, read the actual SLA. Every number that survives into v2 must have a non-LLM source or a "we measured" note. This step is what separates this playbook from vibes with extra steps.

5. 5.

   Publish v2 with the attack visible. Ship: the revised design, a change log ("v2: added Redis-failure open-mode after Lens 1 finding #3"), and the non-goals section. Optionally attach the attack transcript — teams that see the design survived a structured assault review faster and trust deeper. The hinge assumptions become the review's discussion agenda.

Tool variants

• Claude extended thinking / o-series / Gemini: run at least one lens on a second vendor's model — different training produces genuinely different attack surfaces, and the union is broader than either alone.
• Chat with doc attached vs. repo agent: for greenfield designs, chat is fine; when the design touches existing code, a repo agent (Claude Code) grounds attacks in what the current system actually does.
• Persona pass (optional): a final prompt as a specific reviewer archetype your org has ("the DBA who has been burned by unbounded fan-out") catches org-specific blind spots generic lenses miss.

Verification

• •
  Every v1→v2 change traces to a specific finding, and every design-changing claim has a non-LLM source (benchmark, doc page, SLA, measurement).
• •
  The four lens transcripts exist; each produced at least one category-(a) or category-(b) finding (zero findings means the prompt was soft, not the design perfect).
• •
  Non-goals section exists and covers the out-of-scope findings.
• •
  Human review of v2 surfaced no issue the lenses should have caught (post-review, feed misses back into your lens prompts — they compound).

Pitfalls

• Letting the AI redesign instead of attack. "Attack only, no fixes yet" is in the prompt because models rush to solutions — and their redesign anchors you before you have understood the flaw.
• Treating fluent objections as valid ones. Some attacks are confidently wrong about your constraints. Triage category (d) exists so you reject them explicitly rather than absorb them.
• Skipping verification under deadline. The failure mode is specific: a fabricated limit or latency number enters v2, survives review because it sounds precise, and gets discovered in production. Numbers without sources do not ship.
• Running the attack on someone else's design without telling them. As a review aid it is fine — but surprise-AI-audits of a colleague's doc, presented as your own findings, is a trust incident. Disclose the method.

Restricted-environment variant

Design docs encode roadmap, scale numbers, and client architecture — often more sensitive than code. Redact before external prompting: neutralize client/product names, round scale numbers to order-of-magnitude, strip commercial context; the attacks survive redaction because they target structure, not names. On client work, use the sanctioned tenant. Local fallback is honest-but-weaker: an Ollama-hosted model runs the four lenses usably but with shallower reasoning — compensate by running each lens twice with different phrasings and by leaning harder on your own pre-mortem checklist. The steelman step degrades most; consider doing that one as a structured written exercise with a colleague instead.

Success criteria

• •
  v2 differs from v1 in ways you can enumerate, each traced to a lens finding or steelman, with verified numbers.
• •
  The human design review ran shorter and went deeper — arguing judgment calls, not catching omissions.
• •
  At least one hinge assumption got measured and the measurement changed (or deliberately confirmed) the decision.
• •
  The attack-lens prompts live in your team's prompt library and have been reused on a second design.