BXUP Track progress & test yourself — free
Cloud & Solutions Architecture · Playbook · Core · ~4 min read
CLOUD & SOLUTIONS ARCHITECTURE · PLAYBOOK · CORE

Design-Review Red Team — Adversarial Failure-Mode Checklists

Before the design meets a real reviewer, it meets a hostile one: AI armed with a failure-mode checklist — single points of failure, quota walls, cost cliffs, cold starts, data gravity — attacks your architecture, and you triage the hits. Findings are advisory; the triage is the review.

Tier: Core · Hands-on time: ~30 minutes

Every architect has blind spots shaped like their last three systems. An adversarial AI pass is cheap insurance: the model has no stake in your design surviving and pattern-matches against more failure stories than any one career contains. The caution from AI code review carries over directly — a majority of AI review findings can be noise, so this playbook places the red team strictly as an advisory input feeding human triage, never as a gate that blocks or approves anything on its own.

When to use

  • A design is about to enter formal review (ARB, client review board) and you want to find the embarrassing failure first.
  • A high-stakes decision (pb-06 winner, a pb-14 ADR) deserves one hostile pass before commitment.
  • You wrote the design alone and nobody senior is available to challenge it.

Skip AI when the design is still fluid sketch-work — red-teaming a half-formed idea produces findings about things you had not decided yet, and the noise drowns the signal.

Prerequisites

  • The design artifactsHLD or design doc, the verified diagram (pb-07), and key ADRs; the more concrete the input, the sharper the attack.
  • An LLM chat assistant with enough context window for the full set.
  • A failure-mode checklistthe house rubric below, extended with your domain's specials (for AI workloads: prompt injection surface, eval gaps, token-cost runaway).
  • A findings tracker with a disposition column — same discipline as pb-10.

The workflow

  1. Fix the rubric before the attack. A free-form "critique this design" prompt yields generic consulting prose. The named-checklist version forces specificity. House rubric: single points of failure; service limit and quota walls; cost cliffs (pricing-model boundaries where cost jumps nonlinearly); cold-start and warm-up behavior; data gravity and egress traps; blast radius of the riskiest change; failure-mode coupling (two "independent" components sharing a dependency); the 10x question (what breaks first at ten times the load).

  2. Attack with the rubric, demand specificity.

    You are the most adversarial reviewer on a Bank architecture review
    board. Attack the attached design (BFSI document-processing workflow:
    ingestion → OCR → LLM extraction via a gateway → human review queue →
    core-banking write-back). Vikram, the enterprise architect, will triage
    your findings — your job is to make him uncomfortable, not to be right
    about everything.
    
    For each rubric item below, produce findings in this shape:
    - CLAIM: the specific failure scenario, one sentence, naming the
      component(s) from the design.
    - TRIGGER: what condition makes it happen.
    - EVIDENCE: quote the design text/diagram element that exposes it —
      or state ASSUMPTION if the design is silent and you are assuming.
    - BLAST RADIUS: what the user/business sees when it happens.
    Mark any finding that depends on a service limit, quota, or pricing
    claim with [VERIFY] — do not state limits as facts.
    
    Rubric: SPOFs; quota/limit walls; cost cliffs; cold starts; data
    gravity; failure coupling; blast radius; the 10x question.
    ###
    [paste design]
    
  3. Triage every finding — accept, reject, or investigate. Work the list in one sitting. Accept: real, goes on the design's risk register with your severity. Reject: false positive or already mitigated — write the one-line reason, because the rejection log is what proves the red team was a review and not a rubber stamp. Investigate: plausible but unproven — assign the check. Expect heavy rejection; that is the process working, not failing.

  4. Verify the [VERIFY] findings against vendor docs. A red-team finding that "the queue's throughput ceiling will wall you at peak" is only a finding if the ceiling is real, at today's documented value, in your region (pb-05 discipline). Two outcomes, both valuable: the limit is real and you have a design change; the limit is invented and you have calibration data about your model's confidence.

  5. Run the coupled-failure pass separately. One prompt: "List every pair of components in this design that appear independent but share a dependency — a region, an account-level quota, a certificate authority, an identity provider, one team's deploy pipeline. For each pair, quote the shared element." Shared-fate analysis is the red team's highest-yield specialty, and it deserves its own focused pass rather than a rubric line.

  6. Feed survivors forward. Accepted findings become risk-register entries, design changes, or explicit accepted-risk notes in the ADR. The full red-team transcript with dispositions goes into the review pack — arriving at the ARB with "we attacked it first; here is what survived and what we fixed" changes the meeting.

Verify before you trust it

  • Findings cite the actual design. Any CLAIM whose EVIDENCE quote does not appear in your documents is attacking a design the model imagined — reject and note it. The ASSUMPTION tag exists so silence-based findings are labeled, not laundered.
  • No unverified limits survived triage. Sweep accepted findings for [VERIFY] tags that never got their vendor-doc check. An invented quota wall that drives a real design change is the failure mode wearing a safety vest.
  • Rejections have reasons. A disposition column full of bare "rejected" entries will not survive the second question at the ARB. One line each: why it does not apply here.
  • The red team found nothing you already knew to be false. Calibration check: seed one deliberate weakness you know about before the run. If the model missed your planted SPOF, weight its silence on everything else accordingly — absence of findings is not evidence of absence.

Restricted-environment variant

Designs carry client identity and system detail; check the pb-03 register. Redaction preserves attack quality well — failure modes live in structure, not names, so [CORE-BANKING] and [OCR-SERVICE] red-team identically. For BFSI-grade restrictions like Vikram's, run the pass on the bank's own tenancy (Azure OpenAI or Bedrock inside their subscription) or a local Ollama model; the rubric-and-disposition method is model-agnostic and survives smaller models better than most playbooks, since the checklist carries the structure. The triage step is human in every environment.

Remember:

The red team's job is to be prolific, not right; yours is to be right, not prolific. A finding only becomes real when your triage says so — and only stays real if the limits it cites survive the docs check.

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in