Run Automated Garak Probes for a Baseline LLM Vulnerability Scan

Tier: Foundation · Workflow area: AI red-team program & tooling

This playbook stands up a first baseline scan against your own application or a local model you control, so every later hardening step can be measured against a real starting number. It is intentionally the foundation of the red-team track: run it once, understand the report, and you have a defensible "before" snapshot.

When to reach for this

• •
  You are securing an LLM feature your org is shipping and have no security signal on it yet.
• •
  You want an objective baseline before a hardening sprint, so you can prove the sprint worked.
• •
  You are onboarding a new model or provider and want a quick regression against known attack classes.
• •
  Skip this for a one-line prompt tweak — a full scan is minutes of compute and noise you do not need for trivial changes.

The workflow

1. 1.

   Point Garak at a target you control. Use a local model via an OpenAI-compatible endpoint (Ollama, vLLM) or your own app's REST wrapper. Never scan a hosted third-party product.

   Copy

   # Scan a locally served model (Ollama exposing an OpenAI-compatible API)
   python -m garak \
     --model_type openai.OpenAICompatible \
     --model_name llama3.1:8b \
     --generator_option_file ollama_endpoint.json \
     --probes promptinject,leakreplay,dan,encoding
   

2. 2.

   Start with a focused probe set, not everything. The full --probes all run is huge. Begin with the classes that map to your threat model: promptinject (OWASP LLM01), leakreplay and xss (LLM02/LLM05), dan (jailbreaks), encoding (obfuscated payloads).

3. 3.

   Read the run banner and the per-probe pass rate. Garak prints a live table; each line is probe.detector with a pass rate. Below ~90% pass on a probe means the target is failing a meaningful share of that attack class.

   Copy

   promptinject.PromptInjection   detector: promptinject.AttackRogueString   pass 62.0%  (FAIL)
   leakreplay.LiteratureCloze     detector: leakreplay.StartsWith            pass 98.5%  (PASS)
   dan.Dan_11_0                   detector: mitigation.MitigationBypass      pass 71.3%  (FAIL)
   

4. 4.

   Open the HTML/JSONL report. Garak writes a garak.*.report.jsonl and an HTML summary. The JSONL contains every prompt, the model's completion, and the detector verdict — this is your evidence trail and your reproduction set.

5. 5.

   Record the baseline numbers. Capture per-probe pass rates in a tracking sheet or the run's git commit. This snapshot is what every future run in this track (pb-41, pb-46) compares against.

6. 6.

   Triage before you panic. A low pass rate is a lead, not a confirmed exploit. Some detectors are noisy. The interpretation and triage of these findings is its own task (see pb-44) — for the baseline, you only need honest raw numbers.

Verify before you trust it

The scan is trustworthy when it is reproducible and grounded. Re-run the same probe set against the same target twice; pass rates should be stable within a few percent (LLMs are stochastic, so expect minor drift, not swings of 30 points). Then open the JSONL and hand-verify three failing rows: read the actual prompt and the actual model completion, and confirm the detector's verdict is correct — that the model really did emit the rogue string or comply with the jailbreak. If you cannot reproduce a "failure" by pasting the prompt into your app yourself, the detector is a false positive and does not belong in your baseline. A baseline you have not spot-checked is a number you cannot defend to an auditor or a skeptical engineering lead.

Pitfalls

• Scanning something you do not own. The single hard rule. Garak makes it trivially easy to point at any endpoint; keep it aimed at your app, staging, or a local model only.
• Running --probes all on day one. You drown in thousands of results and hours of compute before you understand any of it. Scope to your threat model first.
• Treating detector verdicts as ground truth. Detectors are heuristics; some over-flag. Always spot-check failing rows before reporting a number upward.
• Comparing runs against different model versions or temperatures. A baseline is only meaningful if the target config is pinned. Note the model name, version, and generation settings alongside every run.
• Forgetting stochasticity. One run is a sample. For anything you will act on, average two or three runs so a single unlucky sample does not become a headline finding.

Tools

• Garakthe free, open scanner (NVIDIA) at the center of this task; ~100 probes across injection, leakage, jailbreak, and encoding classes, with machine-readable JSONL output.
• Local model runtimes (Ollama, vLLM)serve a model you fully control behind an OpenAI-compatible endpoint so the scan never touches a third-party service.
• OWASP LLM Top 10the free reference that maps each probe class to a named risk (LLM01 injection, LLM02 disclosure, LLM05 output handling), giving your baseline a shared vocabulary.

Client-safe variant

In a services engagement you can run Garak only against the client's own AI system, and only with written authorization scoped to the specific endpoint — treat it exactly like any other penetration test with a signed rules-of-engagement document. Where client data or endpoints cannot leave their environment, run Garak on-prem or inside the client's VPC against a local or staging model, keep the JSONL report inside their boundary, and share only aggregate pass-rate summaries with names and payloads redacted. Never point the scanner at a client's production customer-facing endpoint without explicit sign-off, and never reuse one client's probe corpus or findings in another engagement.