Use an LLM to STRIDE-model a small AI feature, then run a second, adversarial completeness pass whose only job is to find the threats the first pass missed.
Threat modeling is where LLMs feel most magical and are most quietly incomplete: a first pass produces a confident, well-formatted STRIDE table that looks thorough and silently omits an entire category — usually the AI-specific threats (prompt injection through a tool, data exfiltration via an over-scoped connector) that a generic STRIDE template was never built to surface. You will build the two-pass process that catches this, produce an attack tree for the highest-risk path, and end with a mitigations table mapped to controls. Your artifact is the threat model, the attack tree, and the mitigations table.
A single-pass AI threat model is a checklist that grades its own homework. The completeness gate — a second pass that assumes the first missed something — is what makes it trustworthy.
~90 minutes hands-on · Core tier (maps to module M4 · AI for AppSec, Secure SDLC & Vuln Management).
Offline-capable, $0. Runs on a client-approved endpoint or local Ollama; the structured helper has a --dry-run. Install pip install pydantic (and anthropic if you have a key). No app to deploy — this lab is analysis, and its rigor is in the process, not the code.
The system under review (feature-spec.md — save this):
FEATURE: "OrderHelp" — customer-support assistant on our public website.
- A visitor types a question. An LLM answers, grounded by RAG over our public help-center
articles AND the visitor's own order history.
- The LLM has one tool: lookup_order(order_id) -> returns status, address, items, amount.
- To personalize, the app passes the logged-in customer's session and, if the visitor
pastes an order number, calls lookup_order for it.
- Answers may include clickable links and rendered markdown, shown in the browser.
- Logs of all conversations are stored for quality review.
TRUST BOUNDARIES: browser (untrusted) -> web app -> LLM -> tool -> orders DB.
Note the design already contains a real problem — the assistant mixes private data (order history), untrusted input (whatever the visitor types), and an output channel (rendered markdown/links). Keep that in mind; a good completeness pass will surface it.
Save this two-pass helper as threatmodel.py:
"""Two-pass STRIDE threat model: draft, then adversarial completeness check. --dry-run needs no key."""
import argparse
from pydantic import BaseModel
class Threat(BaseModel):
stride_category: str # Spoofing|Tampering|Repudiation|Info Disclosure|DoS|Elevation
description: str
attack_path: str # concrete steps an attacker takes
impacted_asset: str
likelihood: str # low|medium|high
mitigation: str
class ThreatModel(BaseModel):
threats: list[Threat]
class GapReport(BaseModel):
missed_threats: list[Threat]
reason_first_pass_missed_each: list[str]
def draft(spec: str) -> ThreatModel:
import anthropic
c = anthropic.Anthropic()
return c.messages.parse(model="claude-opus-4-8", max_tokens=3000, thinking={"type": "adaptive"},
system="Produce a STRIDE threat model for the feature. Be concrete about attack paths.",
messages=[{"role": "user", "content": spec}], output_format=ThreatModel).parsed_output
def completeness_gate(spec: str, first_pass_json: str) -> GapReport:
import anthropic
c = anthropic.Anthropic()
return c.messages.parse(model="claude-opus-4-8", max_tokens=3000, thinking={"type": "adaptive"},
system=("You are a skeptical reviewer. The attached STRIDE model is INCOMPLETE. Find threats "
"it missed. Prioritize AI-specific risks: prompt injection via the tool or RAG, "
"excessive agency of lookup_order, data exfiltration through rendered links/markdown, "
"cross-customer data leakage, and log poisoning. For each, explain why a generic "
"STRIDE pass overlooked it."),
messages=[{"role": "user", "content": spec + "\n\nFIRST PASS:\n" + first_pass_json}],
output_format=GapReport).parsed_output
if __name__ == "__main__":
ap = argparse.ArgumentParser(); ap.add_argument("--dry-run", action="store_true")
if ap.parse_args().dry_run:
print(GapReport(missed_threats=[], reason_first_pass_missed_each=[]).model_dump_json())
python threatmodel.py --dry-run prints a schema-valid empty gap report.feature-spec.md). Expect a tidy table: spoofing a session, tampering with the order ID, DoS by flooding, etc. Save as pass1.json. It will look complete. It is not.pass1.json to completeness_gate. The second pass should surface the AI-specific threats the first missed. Save as gaps.json.lookup_order for someone else's order and encodes the result into a clickable link the attacker controls (Info Disclosure + Elevation, AI-flavored). Confirm your gate found this; if it did not, strengthen the gate prompt and rerun.attack-tree.md:GOAL: exfiltrate another customer's order data
├── AND leaf 1: get malicious instructions into the LLM context
│ ├── OR: paste injection text directly in the chat
│ └── OR: poison a help-center article the RAG retrieves
├── AND leaf 2: cause a privileged action
│ └── lookup_order called with a guessed/enumerated order_id
└── AND leaf 3: get the data out
├── OR: render a markdown image whose URL embeds the data
└── OR: emit a clickable link to an attacker domain with data in the query string
mitigations.md with threat · control · layer (input / policy / output / monitoring). For the top path: treat all retrieved/user text as data not instructions (input); scope lookup_order to the authenticated session's own orders and deny cross-customer IDs (policy / least privilege); sanitize and egress-filter rendered markdown/links, strip auto-loading images (output); alert on lookup_order calls for IDs outside the session (monitoring).threat-model.md (merged pass-1 + gap threats), attack-tree.md, and mitigations.md.When an AI threat model omits a category, it is almost always the one the framework has no box for. For AI features that box is "input is instructions." Always ask a completeness pass to hunt AI-specific threats by name.
python threatmodel.py --dry-run passes.attack-tree.md decomposes the top path into get-instructions-in, cause-privileged-action, get-data-out.mitigations.md assigns each top-path control to an input/policy/output/monitoring layer (defense in depth, not a single fix).A threatmodel-lab/ folder with threat-model.md, attack-tree.md, and mitigations.md. Self-grading rubric (pass = 8/10):
Threat models describe client architectures and are sensitive. Run both passes on the client-approved endpoint (Bedrock/Azure OpenAI) or a local Ollama model, and abstract the spec — "an internal LLM with one lookup tool over a customer DB" — rather than pasting real system names, schemas, or connector configs. The two-pass method needs no proprietary detail to work; the deliverable is a model and mitigations table the client can act on without any data having left an approved boundary.
lookup_order on another customer's ID → exfiltration through a rendered markdown image or attacker link. This is Info Disclosure + Elevation with an AI mechanism.