Lab 06 · AI-Assisted Secure Code Review

The lesson is defense-in-depth for review itself: the scanner catches classes the LLM misses, the LLM explains and prioritizes what the scanner floods you with, and you catch what both miss. You will prove that last point concretely: the scanner in this lab has a real false negative (it does not flag a hardcoded secret), and if you trust the tools alone the vulnerability ships. Your artifact is a findings report and a passing regression suite.

Time & tier

~120 minutes hands-on · Core tier (maps to module M4 · AI for AppSec, Secure SDLC & Vuln Management).

Setup

Offline, $0, all local. Install: pip install flask bandit pytest (and anthropic if you have a key; pip install semgrep optional as a second scanner). Create review-lab/ and save this intentionally vulnerable app as app.py. It carries five seeded defects spanning injection, command execution, weak crypto, a missing authorization check, and a hardcoded secret. Do not fix them until your review process finds them.

"""Intentionally vulnerable internal tool. Localhost only. Seeded bugs -- do not deploy."""
import os, sqlite3, hashlib, subprocess
from flask import Flask, request, jsonify

app = Flask(__name__)
API_KEY = "sk-live-internal-4471"                       # BUG 5: hardcoded secret

def db():
    c = sqlite3.connect(":memory:")
    c.executescript("CREATE TABLE users(id INT, name TEXT, role TEXT, ssn TEXT);"
                    "INSERT INTO users VALUES (1,'alice','user','111'),(2,'admin','admin','999');")
    return c

@app.get("/user")                                       # BUG 1: SQL injection (f-string)
def user():
    name = request.args.get("name", "")
    rows = db().execute(f"SELECT name, role FROM users WHERE name='{name}'").fetchall()
    return jsonify(rows)

@app.get("/ping")                                       # BUG 2: command injection
def ping():
    host = request.args.get("host", "")
    return subprocess.check_output("ping -c1 " + host, shell=True)

@app.post("/hash")                                      # BUG 3: weak hash for passwords
def hash_pw():
    pw = request.form.get("pw", "")
    return hashlib.md5(pw.encode()).hexdigest()

@app.get("/admin/ssn")                                  # BUG 4: missing authorization check
def ssn():
    uid = request.args.get("id", "")
    row = db().execute("SELECT ssn FROM users WHERE id=?", (uid,)).fetchone()
    return jsonify({"ssn": row[0] if row else None})

if __name__ == "__main__":
    app.run(port=5002)

Guided steps

1. 1Run the scanner first (cheap, deterministic).bandit -q app.py. It flags the command injection (B602/B605), the weak MD5 (B324), and the subprocess import (B404). Record these. Critically, note what it does not say.
2. 2Notice the scanner's blind spots. Bandit does not flag the f-string SQL injection (it has no taint tracking to know name is user input) and does not flag the hardcoded API_KEY (its secret heuristic keys on variable names like password, not API_KEY). Two of five bugs are invisible to the scanner. Write these two down as "scanner false negatives."
3. 3Run the LLM review to close one gap. Prompt a model: "Review this Flask app for security vulnerabilities. For each, give the line, the vulnerability class (CWE), the exploit in one sentence, and the fix. Rank by severity." The LLM should catch the SQL injection the scanner missed and explain the command injection the scanner flagged but did not contextualize. Save as ai-review.md.
4. 4Do the human pass — catch what both miss. Read the code for authorization, not just injection. /admin/ssn returns any user's SSN to any caller — there is no role check, no session, nothing. Static tools and LLMs both under-detect missing-authorization bugs because the vulnerability is an absence, not a pattern. This is your must-find human catch. Also confirm the hardcoded secret: if the LLM did not surface it, you just proved the point of the lab.
5. 5Build the consolidated findings report.findings.md with columns ID · endpoint/line · class (CWE) · found by (bandit / AI / human) · severity · exploit · fix. The "found by" column is the graded insight — it visualizes the coverage gaps between your three reviewers.
6. 6Write regression tests that fail now. Create test_security.py with pytest cases that exercise each bug as an attacker would, asserting the fixed behavior. Example for the SQL injection:

import importlib
def test_sqli_blocked():
    app = importlib.import_module("app").app
    c = app.test_client()
    r = c.get("/user", query_string={"name": "x' OR '1'='1"})
    # after the fix, injection returns no rows instead of dumping the table
    assert r.get_json() == [], "SQL injection still over-returns"

Add analogous tests: /ping rejects 127.0.0.1; id (no shell metacharacters honored), /admin/ssn returns 401/403 without an admin role, and a test asserting no sk- literal appears in the source (assert 'sk-live' not in open('app.py').read()).

7. 1Run the suite red, fix, run it green.pytest -q test_security.py — expect failures against the vulnerable app. Now apply the fixes (parameterized query; subprocess.run([...], shell=False) with input validation; hashlib.scrypt/bcrypt; an authorization guard; move the key to os.environ). Re-run until green.
8. 2Re-scan to confirm.bandit -q app.py should now be clean of the HIGH findings, and pytest green — but remember the suite, not the scanner, is what proves the SQLi and authz bugs are actually fixed.

Verification

• •
  bandit -q app.py on the original flags command injection and weak hash but not the SQLi or hardcoded secret — you documented both false negatives.
• •
  findings.md lists all five bugs with an accurate "found by" attribution.
• •
  pytest -q test_security.py is red against the original app and green after your fixes.
• •
  The secret is gone from source and the authz check rejects non-admin callers.

Completion artifact

A review-lab/ folder with findings.md, ai-review.md, the fixed app.py, and a passing test_security.py. Self-grading rubric (pass = 8/10):

• Coverage map (0–3): all five bugs found, each correctly attributed to bandit / AI / human = 3.
• Scanner-gap insight (0–2): SQLi and hardcoded-secret documented as scanner false negatives = 2.
• Authz catch (0–2): the missing authorization on /admin/ssn found by human review = 2.
• Regression suite (0–2): tests fail on the vulnerable app and pass after the fix = 2.
• Fix quality (0–1): parameterized query + shell=False + strong hash + env-var secret all applied = 1.

Stuck?

• •
  Bandit reports nothing: you scanned the wrong file or a syntax error aborted it; run python -m py_compile app.py first.
• •
  The LLM "fixes" the code in its review: ask only for findings first; premature fixes hide which reviewer found what.
• •
  Regression tests pass on the vulnerable app: your test asserts the wrong thing — the SQLi test must assert the injection fails (empty result), not that a normal lookup works.
• •
  subprocess.run([...]) still feels risky: it is safe only with shell=False and a validated host; reject anything not matching an IP/hostname pattern.

Client-safe variant

Client source often cannot enter public AI tools. Bandit, Semgrep, and pytest are all local OSS and never see an LLM — run them unchanged. For the AI-review step, use the client-approved endpoint (Bedrock/Azure OpenAI) or a local Ollama model, and review a redacted or representative excerpt rather than pasting proprietary business logic. The regression suite is the durable deliverable: it encodes the vulnerability so it cannot silently return, which is exactly the evidence a client's secure-SDLC gate wants.

Answer key (don't read before attempting)

1. 1BUG 1 — SQL injection (/user, CWE-89): f-string query; x' OR '1'='1 dumps the table. Scanner misses it (no taint tracking); the LLM should catch it. Fix: parameterized ?.
2. 2BUG 2 — command injection (/ping, CWE-78): shell=True with concatenated input. Bandit flags B602/B605. Fix: subprocess.run(["ping","-c1",host], shell=False) + validate host.
3. 3BUG 3 — weak hash (/hash, CWE-327): MD5 for passwords. Bandit flags B324. Fix: bcrypt/scrypt.
4. 4BUG 4 — broken access control (/admin/ssn, CWE-862): no role check; anyone reads any SSN. Neither tool reliably catches it — human find. Fix: require an authenticated admin role.
5. 5BUG 5 — hardcoded secret (API_KEY, CWE-798): bandit's secret rule keys on password-style names and misses API_KEY. Human/AI find. Fix: os.environ["API_KEY"].