A general "review this for security" prompt produces vague hand-wringing. A checklist-driven pass — OWASP/CWE categories mapped to your diff's actual attack surface, findings triaged by exploitability — produces pre-merge catches like the ones Meta reports from their AI review tooling.
Tier: Core · Workflow area: Code Review
The AI is a tireless checklist executor; you remain the judge of exploitability, because that judgment requires knowing your deployment context.
verify=False, weak randomness for tokens) at meaningful rates.This pass supplements, never replaces, your SAST tooling and (for high-risk changes) a human security review.
Run deterministic tools first.
semgrep scan --config auto --json > /tmp/semgrep.json
Anything a rule catches deterministically should never spend LLM tokens or your review attention as a "discovery" — the LLM pass is for what rules cannot see: logic-level and context-dependent flaws.
Run the checklist pass with attack-surface mapping.
You are performing a security review of this diff. Deployment context:
[internet-facing API, JWT auth, handles PII, runs in k8s, Postgres].
Step 1 — Map: list which OWASP Top 10 / CWE categories this diff plausibly
touches, citing the specific lines that create the surface (e.g., "builds
SQL at line 42 → injection/CWE-89"). Say NONE for untouched categories.
Step 2 — For each mapped category, examine the diff and report findings:
- finding, file:line, CWE id
- a concrete attack input or scenario (not "an attacker could...")
- what the code would have to look like to be safe
Step 3 — Separately check these AI-codegen classics: string-concatenated
queries/commands, disabled TLS verification, secrets or tokens in code,
`random` used for security purposes, missing authz check on new endpoints,
path traversal in file handling, unbounded deserialization.
Do not report theoretical issues in unchanged code. Do not pad: if a
category is clean, say clean.
Requiring a concrete attack input per finding is the anti-vagueness lever — a finding without an input is a feeling.
Triage by exploitability, not by CWE severity tables. For each finding, answer three questions the model cannot fully answer for you: Is the input attacker-reachable in our deployment? What is the blast radius given our data and privileges? Are there compensating controls (WAF, gateway authn, network policy)? Sort into: fix-before-merge / fix-this-sprint / accepted-with-note. Write one line of reasoning per accepted finding — that line is your future audit trail.
Verify the fix breaks the attack. For each fix-before-merge item, encode the model's concrete attack scenario as a test:
def test_report_name_is_parameterized():
# attack input from the review finding
resp = client.get("/reports", params={"name": "x' OR '1'='1"})
assert resp.status_code in (200, 400)
assert row_count("reports_audit") == audit_before # no injection side effect
A security fix without a test regresses silently the next time someone "simplifies" the code.
Feed the pattern back. Every real finding becomes either a Semgrep rule (if mechanically detectable) or a line in your AI-review instructions file (pb-18). The goal is that each vulnerability class is caught one layer earlier next time.
LLM security passes have real false-negative rates — a clean pass is evidence, not proof. High-risk surfaces (authn, crypto, payment flows) still get a human security review regardless of what the model says.
.github/copilot-instructions.md so every PR gets them automatically; run the full checklist pass manually on high-surface diffs.SECRET_1 placeholders even in sanctioned tools; report the leak through your incident process, not a chat window.