Generation is cheap; judging what comes back is not. This module gives you a six-part failure taxonomy, verification techniques for facts, code, analysis and documents, a four-tier framework for matching effort to stakes, and three planted-error exercises with an answer key.
Across the fourteen role categories researched for this program — backend, frontend, data, DevOps, security, QA, product, leadership — one advanced skill surfaced independently in every single one: verifying and judging AI output. Not prompting harder, not knowing more tools. Judgment. It is what Meta's AI-assisted interview scores, what Coinbase's mandate legally requires, and what separates the engineer whose AI-accelerated work ships clean from the one whose AI-accelerated defects ship at scale.
Generation is now cheap and universal; 97% of developers use AI tools. Verification is scarce, and scarcity is what markets pay for. This is the flagship module of the Common Core: a failure taxonomy so you know what you're hunting, techniques per output type so you know how, a trust-tier framework so you know how much, and three planted-error exercises so you know whether you can actually do it.
In 2026 the deliverable is no longer the draft — AI produces drafts on tap. The deliverable is the judgment that the draft is correct, safe, and fit for purpose. That judgment is the skill employers interview for, mandate, and pay premiums to get.
Follow the evidence chain from this program's research base:
The assessment framing to internalize: when AI writes the code, the human's remaining job is the review. If you sign the PR, the merge, or the client email, the defects are yours — "the AI wrote it" has exactly the professional standing of "the intern wrote it and I didn't read it."
You cannot catch what you cannot name. Six recurring failure modes cover the vast majority of AI output defects. Learn them as a checklist you run mentally against every output that matters.
The model invents an entity that does not exist: an API parameter, a library function, a court case, a statistic, a citation, a config option. Fabrications are formatted exactly like real ones — that's what "plausible continuation" means. Tell: specifics that arrive conveniently — the perfect function for your exact need, the statistic that lands on a round number, the reference you can't immediately locate. Highest risk where your own knowledge is thinnest.
Everything referenced is real; the claim connecting them is false. The API exists but doesn't behave as described; the regulation exists but says something different; the design pattern is real but misapplied to your case. Harder to catch than hallucination because spot-checking the nouns passes — you must check the verbs.
Correct at training time, wrong now: deprecated APIs, superseded library versions, old pricing, pre-amendment regulations, last year's org chart. The model rarely volunteers its cutoff unprompted, and stale answers carry no staleness marker. Tell: any answer about fast-moving dependencies, current events, prices, or versions that doesn't come from material you supplied.
The output quietly answers a different question than you asked — broader, narrower, or adjacent. You asked for the fix to one function and got a refactor of the file. You asked about your Postgres 14 and got generic SQL advice. You asked for Q1 findings and got claims about metrics the data doesn't contain. Drift often adds correct-looking material, which makes it feel like generosity rather than error.
The model optimizes for your approval: it agrees with the framing embedded in your question, softens criticism you didn't ask it to soften, and folds when you push back on a correct answer. Ask "this design is solid, right?" and you will usually hear yes. Sycophancy corrupts exactly the workflow professionals most want AI for — a second opinion — unless you engineer the prompt to authorize disagreement and verify the agreement you receive.
The taxonomy's signature member — the 66% frustration. Code that compiles, passes the happy path, reads cleanly, and is wrong: an edge case unhandled, a race introduced, an error swallowed, a contract in the docstring the body doesn't honor, a subtly wrong boundary condition. Almost-right code is more dangerous than broken code because broken code fails in front of you and almost-right code fails in front of your customers.
Run the six as a scan — invented? mis-claimed? outdated? off-target? flattering me? almost-right? The taxonomy's value is speed: naming the likely failure mode tells you where to point your verification effort instead of vaguely re-reading.
Verification is not one skill but a small kit, keyed to what you're looking at.
The discipline is read, run, test — in that order, skipping none:
Verification effort should be targeted, not uniform. The taxonomy tells you where to aim: unfamiliar-domain output → hunt hallucination first; fast-moving tech → staleness first; numbers → recompute first; anything you were hoping to hear → sycophancy first.
Full verification of everything would erase AI's speed advantage; zero verification is malpractice. The professional answer is calibration — matching verification depth to stakes. Three questions set the stakes: What's the blast radius if this is wrong? How reversible is the mistake? Can I actually verify this — do I have the expertise and the source material? That last one is a hard constraint: if you cannot verify it, you cannot delegate it at high stakes, no matter how good the output looks.
| Tier | Rule | Verification required | Typical examples |
|---|---|---|---|
| T1 — Draft-only | AI output is raw material; you rework it, so review is inherent | Editorial pass; no formal audit | Brainstorms, first drafts you'll rewrite, throwaway scripts, learning explorations, naming ideas |
| T2 — Verify key claims | Ship after checking the load-bearing elements | Claim audit on facts/numbers/APIs that matter; skim the rest | Internal docs, routine emails, low-risk utility code with tests, meeting summaries you attended |
| T3 — Full-verify | Every claim checked, every line read, tests run, before it ships | Complete read-run-test / full claim audit / recompute pass | Production code, client deliverables, published content, compliance-adjacent text, data behind decisions |
| T4 — Never-delegate | AI may inform your thinking; the artifact is yours, made by you | N/A — you produce it; AI is at most a sounding board | Legal/contractual commitments, security-critical logic, personnel decisions, safety-relevant configs, anything you sign under professional or regulatory liability |
Two disciplines make the framework real:
The most dangerous tier transition is silent T1→T3: the exploratory snippet that gets pasted into production, the "rough summary" that gets forwarded to the client. Tier travels with the artifact's destination, not its origin. Re-verify at the destination's tier, every time.
Individual discipline decays under deadline pressure; teams that keep AI's speed advantage without its defect tax institutionalize verification as gates — the same move as code review and CI, extended to AI-mediated work. DORA's amplifier finding applies directly: teams with strong review practice get faster with AI; teams without it generate defects faster. Elements that recur in mature policies (Coinbase's being the publicly documented reference):
Verification gates are also the bridge to a later Common Core module: evals are verification industrialized — turning the manual judgments this module trains into automated, repeatable checks that run on every change. Teams graduate from "humans check everything" to "humans design the checks."
Three realistic AI outputs follow, each containing planted errors drawn from the taxonomy. For each: find the errors, name the taxonomy class of each, and state which verification technique catches it. Work before reading the answer key. Scoring guide: finding all major planted errors with correct technique = interview-ready; finding half = you now know which sections to reread.
A colleague asked an AI assistant to summarize data-protection obligations for a client proposal. Audit it — some statements are correct, some are not:
Under GDPR, organizations must report qualifying personal data
breaches to their supervisory authority within 30 days of becoming
aware of them. Serious violations carry fines of up to €20 million
or 2% of global annual turnover, whichever is higher. Note that
GDPR obligations only apply to companies that maintain an office
or legal entity within the EU. For Indian operations, the Digital
Personal Data Protection Act of 2021 applies, with penalties for
significant breaches reaching ₹250 crore per instance.
A coding assistant produced this for the task "fetch all users from our paginated API, with retries." Review it as if it were headed for production (T3):
import requests
import time
def fetch_all_users(base_url, max_retries=3):
"""Fetch every user from the paginated /users endpoint.
Retries each failed request with exponential backoff and
raises RuntimeError if a page still fails after max_retries.
"""
users = []
resp = None
page = 1
total_pages = 1
while page <= total_pages:
for attempt in range(max_retries):
try:
resp = requests.get(
f"{base_url}/users",
params={"page": page},
timeout=5,
)
resp.raise_for_status()
break
except requests.RequestException:
time.sleep(2 ** attempt)
payload = resp.json()
users.extend(payload["users"])
total_pages = payload.get("total_pages", 1)
page += 1
return users
Raw deployment data for Q1:
| Month | Deploys | Failed deploys |
|---|---|---|
| Jan | 40 | 4 |
| Feb | 50 | 6 |
| Mar | 60 | 3 |
The AI's executive summary:
Q1 was a strong quarter for delivery reliability. The team shipped
150 deploys — an average of 55 per month — with only 13 failures,
an overall failure rate of 8.7%. The monthly failure rate improved
steadily throughout the quarter, falling from 10% in January to 5%
in March. Deploy volume grew 50% from January to March, and mean
time to recovery improved alongside it, confirming that the new
release process is paying off.
Exercise 1 — four planted errors:
Exercise 2 — one critical bug, two secondary findings:
max_retries attempts for a page fail, the for loop simply ends — no RuntimeError is ever raised, despite the docstring's promise. Execution falls through to resp.json(), where resp is either None (crash with a misleading AttributeError if page 1 fails) or — far worse — the previous page's successful response, so the previous page's users are silently appended again and the failed page's users are silently lost. The function returns corrupted data with no error. Caught by reading the verbs: checking that the body actually implements the docstring's contract. Fix: a for/else clause (or a raise after the loop) so exhausted retries raise RuntimeError as promised.raise_for_status() failures (401, 403, 404) are caught by the same except requests.RequestException and retried with backoff — three slow, pointless attempts against a deterministic client error. Retry logic should distinguish retryable (timeouts, 5xx, 429) from non-retryable failures.payload["users"] will KeyError on an unexpected response shape while total_pages defensively uses .get — inconsistent defensiveness worth flagging in review.Exercise 3 — three planted errors (and deliberately, several correct numbers):