This module installs the three-actor habit of modern AppSec: let the scanner reason about sinks, let AI reason about intent, and confirm the exploit yourself before anyone files, fixes, or ships on either one's word.
This module puts AI to work across the AppSec chores that eat your week: reviewing code (yours and the AI's), threat-modelling features, cutting scanner noise, prioritising vulns by what an attacker can actually reach, and pinning fixes with regression tests. Every technique lands on a control you can run in CI, and every AI conclusion is treated as a hypothesis you verify against running code — never a verdict you ship on the model's word alone.
Rohan is the AppSec engineer for a 40-developer product company; nine years in, ISO 27001 and SAST are his comfort zone, LLM-authored code is not.
A PR lands in orders-service: feat: export order history to CSV, +214 / −6 lines, authored half by a developer and half by Copilot. Semgrep runs in CI, returns green, and the PR has one approving review. It is exactly the kind of change that used to sail through.
Rohan pastes the diff into an internal review assistant running Claude behind the company's own gateway, with a prompt that asks one thing: "For each new route, state who is authorised to call it and cite the line that enforces it."
GET /orders/:id/export reads order_id straight from the path and queries SELECT * FROM orders WHERE id = :id — no WHERE user_id = :current_user. The AI-generated code had faithfully mirrored the adjacent getOrder handler, which relied on an ownership check in a middleware the new route didn't inherit. A textbook IDOR: any authenticated user could export any other user's orders by incrementing the ID.
There was no dangerous sink — no eval, no raw SQL string concat, no innerHTML. The query was parameterised and clean. The vulnerability was missing authorization, a semantic gap a pattern scanner cannot see and a busy human reviewer skimmed past because the code looked like its safe neighbour.
Rohan does the part the AI cannot: he opens the middleware, confirms the ownership check really is absent on this route, and reproduces the access with a second test account — curl with user B's token against user A's order_id, HTTP 200, wrong data returned. The finding is real, not a hallucination.
He does not merge the AI's suggested fix blindly; he reads it, confirms it adds AND user_id = :current_user and returns 404 (not 403) to avoid leaking existence, and asks the assistant for a regression test that fails on the old code and passes on the new.
The PR carries a failing-then-passing test_export_denies_cross_tenant_access, a one-line Semgrep custom rule to catch the pattern next time, and a merged fix. Time from AI flag to shipped control: 44 minutes. Time the same class of bug spent undiscovered in their last incident: five months.
The lesson Rohan writes in the ticket is the module in one line — the scanner tells you about sinks, the AI reasons about intent, and the human confirms the exploit before anyone trusts either.
Start with the prompt, not the principle. A review that asks the model to "find bugs" produces a wall of low-signal style nits; a review scoped to a threat question produces the IDOR above.
You are reviewing a diff for security defects only. For every new or changed
route/handler:
1. State the authorization required to call it and QUOTE the line that enforces it.
2. If no line enforces it, output: MISSING AUTHZ + the route.
3. Flag any user-controlled value that reaches a sink (SQL, shell, path, HTML,
deserialization, SSRF-capable fetch) and quote both the source and the sink line.
Do not report style, naming, or performance. Cite line numbers. If unsure, say UNSURE.
The principle behind it: AI-generated code fails differently from human code. It rarely produces the obvious os.system(user_input) a scanner catches; it produces plausible-but-unauthorised code that mirrors nearby patterns while dropping an invariant — the missing ownership check, the copied-but-now-wrong CORS config, the hardcoded fallback secret. Phase-1 data shows roughly half of engineering roles now review AI-written code, and mandated human review (Coinbase-style) exists precisely because review is where these semantic gaps get caught. The volume shift is the real risk: an organisation shipping AI-authored code moves faster than its review capacity was sized for, so the review itself has to get faster without getting shallower — which is exactly the leverage an intent-scoped AI reviewer provides, catching the semantic gaps at machine speed so the human reviewer spends their attention on confirmation rather than first-pass reading.
Treat the AI reviewer as a first-pass reasoner over intent that a SAST tool cannot match, and yourself as the verifier. Two to three tools cover the capability vendor-neutrally:
The AI finds what the scanner can't reason about; the scanner finds what the AI overlooks; you decide what is real.
The failure modes worth memorising, because they recur in every AI-generated diff:
verify=False, a wildcard origin, httpOnly forgotten. These pass review because they resemble correct config.if env == 'dev': skip_auth branch that ships to prod.None of these are exotic; they are ordinary AppSec bugs arriving faster and looking more legitimate than before. That is precisely why the review discipline matters more, not less, as AI-authored code volume climbs.
Scanners reason about sinks, AI reasons about intent, humans confirm exploitability — use all three and trust none of them alone.
An AI reviewer will confidently invent a vulnerability that isn't there (a false positive) as readily as it misses one — never file, escalate, or fix a finding you haven't reproduced against running code.
Give the model the design and a schema, not an open question.
Threat-model this feature using STRIDE. Output one row per threat:
{category, component, threat, precondition, existing_control (quote it or say NONE),
residual_risk (H/M/L), mitigation}. Then run a SECOND pass: list threats a
STRIDE-by-the-book review commonly misses here (trust boundaries between services,
token replay, cache poisoning, the AI feature's own prompt/output surface).
The two-pass structure matters: the first pass gives coverage, the second pass hunts the model's own blind spots — the completeness check is what turns a generic STRIDE table into a useful one. For features that ship an LLM or agent, the threat model must include the AI surface itself (prompt injection, insecure output handling), which lands you in Modules 5 and 6. Tools: any LLM plus a STRIDE / attack-tree template; keep the output as a checked-in artifact so the model reviews diffs to the threat model as the design evolves.
Where AI genuinely accelerates threat modelling is breadth without fatigue: a human modeller runs out of patience around the fifth data flow, but the model will enumerate spoofing and tampering threats for every trust boundary at the same depth. Where it fails is depth of your context — it doesn't know that your auth service is the one component with a known replay weakness, or that a particular queue is internet-reachable through a misconfigured load balancer. So the division of labour is: the model produces the exhaustive first draft and the completeness sweep; you inject the environment-specific knowledge and prune the threats that don't apply to your deployment. Feed it an attack-tree prompt for the highest-risk threat — "given the injected secret in the CI file, enumerate the paths an attacker could take from an issue comment to that secret" — and it will build a tree you can then validate branch by branch.
A threat model that omits the trust boundary between two of your own services is incomplete — STRIDE's Spoofing and Tampering categories live exactly at those boundaries, and the second (completeness) pass exists to surface them.
A SAST run of 300 findings is not 300 bugs — it is mostly noise, and the noise is why real findings get ignored. This is not a productivity annoyance; it is a security problem in its own right. When a queue is 90% false positives, developers learn to dismiss the tool, and the 10% that are real get dismissed along with the rest — alert fatigue is how genuine vulnerabilities ship. The goal of AI-assisted triage is therefore not merely to save time; it is to restore signal so that a finding that survives triage is one people actually act on. The failure mode to avoid is an AI that suppresses findings by guessing they're benign — that just trades a noise problem for a silent-miss problem, which is worse. Constrain it to reason from reachability, and make the decision auditable.
Route each finding through a structured triage that returns a decision plus its justification, and — critically — errs toward NEEDS_HUMAN rather than silently closing anything:
from pydantic import BaseModel
from typing import Literal
import anthropic
class Triage(BaseModel):
verdict: Literal["true_positive", "false_positive", "needs_human"]
reachable_from_untrusted_input: bool
justification: str # must cite the data path or the control that blocks it
suggested_regression_test: str
client = anthropic.Anthropic()
def triage_finding(rule_id: str, code_context: str, data_flow: str) -> Triage:
resp = client.messages.parse(
model="claude-opus-4-8",
max_tokens=2000,
thinking={"type": "adaptive"},
system=(
"You triage SAST findings. Mark false_positive ONLY if you can name the "
"control that makes the sink unreachable from untrusted input, and quote it. "
"If you cannot prove reachability either way, return needs_human. Never "
"suppress on a guess."
),
messages=[{
"role": "user",
"content": f"Rule: {rule_id}\nCode:\n{code_context}\nData flow:\n{data_flow}",
}],
output_format=Triage,
)
return resp.parsed_output
Everything the model calls false_positive carries a named, quotable control; everything it can't prove goes to a human. That inversion — suppress only with proof, escalate on doubt — is what makes AI triage safe to wire into CI. Tools that fit: Semgrep or CodeQL to generate findings, an LLM for the reachability reasoning, and your issue tracker for the audit trail. Never let the model close a finding without the justification field populated.
Log every AI suppression with its justification and re-review a sample monthly — a suppression you can't audit is indistinguishable from a bug you shipped.
CVSS tells you a vuln's theoretical severity; it does not tell you whether anyone is exploiting it or whether the vulnerable code path is even reachable in your deployment. A queue sorted by CVSS puts a 9.8 in an unreachable dev dependency above a 6.5 being mass-exploited in the wild. Fix the sort key.
Feed the model three signals and have it reason over context, not just score:
Rank these CVEs for remediation. For each, combine:
- CVSS base (theoretical severity)
- EPSS score (probability of exploitation in the next 30 days)
- CISA KEV membership (known exploited in the wild — treat as top priority)
- our context: is the affected component internet-facing? is the vulnerable
function actually called in our code? (I provide call-graph evidence)
Output: ranked list with a one-line rationale per item citing which signal drove
the rank. Do NOT rank on CVSS alone.
The discipline: KEV membership and a reachable, internet-facing path beat a high CVSS every time. EPSS gives you exploitation probability; KEV gives you exploitation fact; your call-graph gives you reachability. An AI that blends these into a ranked queue with per-item rationale turns vuln management from severity-theatre into a defensible triage — but only if you verify the reachability claims, because "the vulnerable function isn't called" is exactly the kind of confident assertion that is sometimes wrong. Tools: EPSS and the CISA KEV feed (both free), an LLM for the synthesis, and your CMDB/SBOM for the asset and call-graph context.
A worked example makes the reordering concrete. Suppose your scanner surfaces three findings: a CVSS 9.8 deserialization bug in a build-time dev dependency that never runs in production, a CVSS 6.5 auth-bypass in your internet-facing API gateway that sits on the CISA KEV list with an EPSS of 0.71, and a CVSS 8.1 in an internal library whose vulnerable function your call-graph shows is never invoked. A CVSS-sorted queue patches the 9.8 first, the 8.1 second, and the actively-exploited 6.5 last. The exploitability-aware ranking inverts it: the KEV-listed, internet-facing, high-EPSS 6.5 is the fire, the 9.8 is a low-priority hardening item because it is unreachable, and the 8.1 drops near the bottom because the vulnerable code path is dead. This is also where CVE impact analysis against your asset inventory pays off — a RAG query over your CMDB answers "which of our systems actually run the affected version?" so you patch the twelve hosts that matter instead of chasing a CVE across an inventory it doesn't even touch. The remediation guidance and patch communications — the advisory to the affected teams, the customer-facing note — are themselves a fast AI-drafting task, grounded in the structured facts you've just assembled rather than written from scratch.
A vuln on the CISA KEV list is being exploited right now — it outranks a higher-CVSS finding that no one has ever weaponised, regardless of what the base score says.
A fix without a regression test is a fix that will be reintroduced. After the IDOR in the scenario, the durable artifact was not the patch — it was test_export_denies_cross_tenant_access, which fails on the vulnerable code and passes on the fixed code. Ask for exactly that property:
Write a security regression test for this fix. It MUST:
- fail against the PRE-fix code (prove it catches the vuln), and
- pass against the POST-fix code.
Test the security property directly (user B cannot read user A's resource),
not the implementation. Give me the test plus the command to run it against both revisions.
The "fails before, passes after" requirement is what distinguishes a real security test from a test that merely exercises the happy path. Run it against both revisions to prove it — a test that passes on both the vulnerable and fixed code is testing nothing about the vulnerability, and a surprising number of AI-generated "security tests" are exactly that: they assert the endpoint returns 200 for a valid request without ever checking that it returns 404 for an unauthorised one. The failing-first step is not a formality; it is the proof that the test would have caught the original bug.
Over time these accumulate into a security regression suite — the Applied-Materials-style harness that turns each incident into a permanent guardrail. Test the property, not the implementation: user B cannot read user A's resource survives a refactor, whereas a test that asserts a specific middleware is present breaks the moment someone moves the check. The same pattern extends to IaC and cloud config: an LLM plus Checkov / tfsec / Trivy explains a misconfiguration finding — why an over-permissive IAM policy or a public S3 bucket is dangerous, in terms the developer can act on — and generates a policy-as-code test that fails on the bad Terraform and passes on the fixed version. Cloud misconfiguration is one of the most common real-world breach causes, and a checked-in policy-as-code regression is what stops the same 0.0.0.0/0 security-group rule from reappearing in next quarter's module.
Extend the same discipline to infrastructure — a tfsec finding on an over-permissive S3 policy deserves a policy-as-code regression test exactly as much as an application IDOR does.
For services engineers on client engagements, the hard limit is that client code, logs, and PII often cannot be pasted into public AI tools. Four steps keep you both productive and compliant:
The techniques in this module are endpoint-agnostic — the diff-review prompt, the triage schema, the EPSS/KEV ranking all run identically against a local model; only the endpoint and the redaction step change. That dual competence — AI-assisted AppSec and client-data discipline — is directly billable and is a gap most competitors don't cover. Build the redaction and endpoint-selection step into your workflow as muscle memory, not an afterthought: the moment a prompt containing client code goes to a non-approved endpoint, the productivity gain becomes a compliance incident, and no finding is worth that trade.
You now have five AI-assisted AppSec moves, each ending in a control rather than a suggestion — all runnable against a client-approved or local endpoint:
Take these into Lab 06 (AI-assisted secure code review), where you'll review an app mixing human- and AI-generated code, find the injected bugs, and generate the regression suite that pins them down — and Lab 07 (threat-modelling assistant), where you'll STRIDE-model a small AI feature and run the completeness check that hunts the threats the first pass missed. Bring the triage schema and the "fails-before, passes-after" test requirement with you; both are graded artifacts.
Your first 15 minutes: on the next PR you review, ask your AI assistant one question per new route — "who is authorised to call this, and which line enforces it?" — and if it cannot cite the enforcing line, assume the authorization check is missing until you have opened the middleware yourself.
Check yourself: Semgrep runs green on a PR and a reviewer approves it, yet the new endpoint lets any authenticated user read any other user's data. Why did the scanner miss it, and what does each of the three actors — scanner, AI, human — contribute to catching it?
Companion playbooks — take these into your week alongside Lab 06 and Lab 07: pb-14 (Review Human- and AI-Generated Code for Vulnerabilities with AI), pb-16 (Triage SAST/DAST Findings and Cut the False-Positive Pile with AI), and pb-20 (Prioritize Vulnerabilities by Real Exploitability, Not Raw CVSS).