Adversarial Design Review: API Contracts and Schema Migrations

Tier: Core · Workflow area: Feature development

This playbook is a two-move pattern: draft the artifact (yourself or with AI), then have AI adversarially critique it against a domain checklist before any human review. It works identically for API contracts and DB schema/migration designs.

When to use

• •
  Designing or extending a public/internal API (REST, OpenAPI, gRPC/proto).
• •
  Any migration on a large or hot table: new columns, backfills, index changes, type changes.
• •
  Before design review — arrive with the obvious attacks already answered.

Skip it for additive changes to internal-only endpoints with one consumer you control.

Prerequisites

• •
  A drafted artifact: OpenAPI/proto snippet, or CREATE TABLE/ALTER TABLE DDL plus migration steps.
• •
  Real context numbers: row counts, traffic, p99 latency budget, database engine and version. Attacks without numbers are generic.
• •
  A chat assistant; no repo access needed (the artifact is the context).

The workflow

1. 1.

   Draft with explicit context. Paste the artifact plus facts the attacker needs:

   Copy

   Context: Postgres 15, orders table ~40M rows, ~600 writes/min peak,
   API consumed by 3 internal services + 1 mobile app we cannot force-upgrade.
   
   Draft artifact: [OpenAPI for GET/POST /v2/orders + ALTER TABLE orders
   ADD COLUMN fulfillment_status text NOT NULL DEFAULT 'pending';]
   

2. 2.

   Run the API attack. One prompt, checklist-driven:

   Copy

   Attack this API design as a hostile senior reviewer. For each finding:
   the flaw, a concrete failure scenario, and the minimal fix.
   Checklist: pagination (what happens at 100k orders?), filtering/sort
   stability, idempotency of POST (mobile retries!), error taxonomy
   (structured codes vs free text), versioning and additive-change policy,
   nullable-vs-optional semantics, timestamps and timezones, rate-limit
   headers, breaking-change risk for the mobile client we cannot upgrade.
   Rank findings by blast radius. Do not restate the design.
   

3. 3.

   Run the migration attack. Same pattern, different checklist:

   Copy

   Attack this migration for Postgres 15 at 40M rows / 600 writes-min.
   Checklist: lock type and duration of each statement (which ones take
   ACCESS EXCLUSIVE?), table rewrite risk of NOT NULL DEFAULT on this PG
   version, backfill strategy (batch size, throttling, resumability),
   index creation (CONCURRENTLY? invalid-index cleanup?), deploy ordering
   vs old app code still running, rollback plan per step, replication lag.
   For each: how I would detect the problem in staging before prod.
   

4. 4.

   Adjudicate the findings. For each finding, mark: accept (fix the design), reject with reason (write the reason down — it becomes review documentation), or verify (you are unsure; test it). Never accept fixes wholesale — attackers over-flag by design.

5. 5.

   Verify the load-bearing claims. Lock behavior and rewrite claims are version-specific and models get them wrong across versions. Check the two or three decisions that matter against the official docs or an empty-table experiment (BEGIN; ALTER ...; SELECT mode FROM pg_locks ...; ROLLBACK;).

6. 6.

   Attach the record. Bring the attack findings and your adjudications to human design review. Humans now spend their time on the judgment calls, not on re-finding the missing pagination.

Tool variants

• General chat assistant (Claude, ChatGPT, Gemini): ideal — this is pure artifact reasoning; use the strongest reasoning model you have.
• Copilot Chat in the IDE: convenient when the OpenAPI/migration files live in the repo; reference the files directly.
• Two-model cross-examination: run the same attack in two different vendors' models and diff the findings — disagreements are your "verify" list.

Verification

• •
  Every accepted finding produced a concrete diff to the artifact.
• •
  Every rejected finding has a one-line written reason.
• •
  Version-specific claims (locks, rewrites, default behavior) checked against official docs or a live experiment — at least the top two.
• •
  Migration rehearsed on a staging copy with production-like row count before prod.

Pitfalls

• Attacking without numbers. Without row counts and traffic, you get boilerplate advice ("consider indexing") instead of "this specific statement rewrites the table".
• Accepting the attack as a to-do list. Some findings are wrong or irrelevant to your consumers; adjudication is the skill being practiced.
• One mega-prompt for API + schema together. Interleaved findings get shallow. Run the two attacks separately.
• Skipping the rehearsal. A clean attack transcript is not a tested migration.

Restricted-environment variant

Schema and API shapes are often exactly what NDAs protect. Sanitize before pasting: rename tables/fields to neutral equivalents (orders → records, keep types and cardinalities honest — the attack quality depends on structure, not names), strip real hostnames and internal service names, and keep a local mapping note. Or run the whole attack on a client-approved enterprise tenant or a local model — attack checklists work well even on mid-size local models because the prompt itself carries the expertise.

Success criteria

• •
  At least one design change you would not have made otherwise, caught pre-review.
• •
  Human design review focused on trade-offs, not omissions.
• •
  The migration ran in production inside its maintenance window with no lock incident.
• •
  Your adjudication notes answered a reviewer question before they asked it.