Put Unbounded-Consumption and Cost-DoS Limits on Your LLM App

Tier: Foundation · Workflow area: LLM attack surface & defense

Capping consumption is a Foundation control because the fixes are ordinary rate-limiting and quota discipline applied at the right layer. This playbook adds those limits to your own app and shows how to load-test that they actually bite.

When to reach for this

• •
  Any LLM feature exposed to users, partners, or the internet with a metered model behind it.
• •
  Long-context or agentic features where one request can fan out into many model calls.
• •
  Finance or platform asks "what's the worst-case spend if this is abused?" and you don't have a number.

The workflow

Threat, briefly. An attacker (or a buggy client) sends huge inputs, requests enormous outputs, loops an agent, or floods the endpoint. Each call costs tokens and latency; unbounded, it exhausts your quota, spikes your bill, and starves legitimate users.

1. 1.

   Cap input and output size per request. Reject oversized inputs before they reach the model and bound max_tokens to what the task genuinely needs.

   Copy

   MAX_INPUT_CHARS = 20_000
   if len(user_input) > MAX_INPUT_CHARS:
       raise ValueError("input too large")   # fail fast, pre-model
   
   response = client.messages.parse(
       model="claude-opus-4-8",
       max_tokens=1500,                       # bounded, not open-ended
       thinking={"type": "adaptive"},
       messages=[{"role": "user", "content": user_input}],
       output_format=Answer,
   )
   

2. 2.

   Rate-limit per identity, not just per IP. Apply token-bucket limits keyed on the authenticated user or API key so one caller cannot monopolize the endpoint.

   Copy

   # sliding-window / token-bucket at the gateway; pseudocode
   if not limiter.allow(key=user_id, cost=estimated_tokens):
       return http_429("rate limit exceeded")
   

3. 3.

   Set hard spend and concurrency quotas. Enforce a per-user and per-tenant daily token/spend ceiling and a max concurrent-request count, with alerts well before the ceiling.

4. 4.

   Bound agent loops. Agentic features multiply cost per user turn. Cap tool-call iterations and total tokens per session so a goal-hijack or a loop can't run forever (this pairs with the excessive-agency playbook).

   Copy

   MAX_STEPS = 8
   for step in range(MAX_STEPS):
       ...            # break on completion
   else:
       abort("step budget exhausted")
   

5. 5.

   Monitor and alert on consumption. Track tokens/spend per user over time; a sudden spike is your earliest abuse signal. Wire a threshold alert to on-call.

6. 6.

   Load-test the limits on your local app. Fire a burst and an oversized-input flood at your local instance and confirm the limiter returns 429s and the spend meter plateaus instead of climbing.

   Copy

   # local target only
   seq 1 500 | xargs -P50 -I{} curl -s -o /dev/null -w "%{http_code}\n" \
     localhost:8080/ask -d '{"q":"hi"}' | sort | uniq -c
   # expect a wall of 429 once the bucket drains
   

Verify before you trust it

The defense holds when abuse is capped and observable. Run three checks against the local app:

• Oversized input rejected: an oversized input is rejected pre-model with no token spend — confirm the model was never called.
• Per-identity throttling: a burst from one identity gets throttled to 429 after the bucket drains, while a second identity is unaffected, proving per-identity fairness.
• Step budget holds: an agent given an impossible goal halts at the step budget instead of looping, and the session's total token count is bounded.

Watch the spend meter throughout — the passing result is a flat line under load, not a climbing one. Keep these as automated checks so a removed limit fails CI.

Pitfalls

• IP-only rate limits. Trivially bypassed with rotating IPs and unfair to users behind shared NAT. Key on identity.
• Unbounded max_tokens. Leaving output length open invites maximal-cost responses on every call. Bound it to the task.
• No agent step cap. A single agentic request can silently become hundreds of model calls; without a loop bound, cost is unbounded by design.
• Limits without alerts. A quota that only surfaces on the invoice is a postmortem, not a control. Alert before the ceiling.
• Global limits that starve everyone. A single shared bucket lets one abuser DoS all users. Partition per tenant.

Tools

• An API gateway with rate limiting (Kong/Envoy/APISIX-class or your provider's gateway) — token-bucket and quota enforcement outside the app code.
• An LLM gateway / proxy (LiteLLM-class) — per-key spend caps, budgets, and usage tracking across model providers.
• Metrics + alerting (Prometheus/Grafana-class) — per-identity token and spend dashboards with threshold alerts.

Client-safe variant

For client deployments the limits protect the client's budget and their SLA, so agree the per-tenant ceilings and burst limits with them up front and enforce them in the gateway that sits inside their environment. The load test runs against a client-approved staging instance, never production, using synthetic traffic. None of this requires sending client data anywhere — you are testing capacity and throttling, not content — so the entire exercise fits inside an air-gapped or on-prem setup. Hand the client a one-page worst-case-spend estimate and the limit values that cap it, which doubles as evidence of a resource-abuse control for their risk register.