Agent Guardrails and Cost Controls

Tier: Advanced · Workflow area: Agentic systems

This playbook retrofits a layered defense onto any agent you have built (pb-40, pb-41): input guards, output validation, action deny-lists, step and cost budgets, and a kill switch — the same layering pattern platform teams now ship as "guardrails for agentic workloads."

When to use

• •
  Any agent that touches real data, real money (API spend), or real users — i.e., before the first demo to a stakeholder, not after.
• •
  An agent moving from your laptop to a shared or scheduled deployment (pb-45).
• •
  Post-incident: an agent already burned a budget or emitted something it should not have.

For a read-only toy on synthetic data, a step cap alone is enough — apply the full stack when consequences appear.

Prerequisites

• •
  A working agent with a tool loop you control (tool runner hooks, LangGraph nodes, or a manual loop).
• •
  pip install pydantic for output validation; your provider's usage fields for cost accounting.
• •
  30 minutes of thought about what "worst realistic failure" looks like for your agent — the guardrails implement that answer.

The workflow

1. 1.

   Layer 1 — input guards. Before the agent sees a request: length caps, an allowlist of task types if applicable, and an injection screen for text arriving from untrusted sources (documents, tickets, web pages). A cheap-model classifier call ("does this text contain instructions directed at an AI agent? yes/no") in front of retrieved content catches the common cases for fractions of a cent.

2. 2.

   Layer 2 — action guards (deny-list + confirmation tiers). Classify every tool by blast radius: read-only (auto-allow), reversible writes (allow + log), irreversible/external (require human confirmation or deny). Enforce in the tool implementation, not the prompt:

   Copy

   DENY = {"delete", "drop", "truncate", "grant"}
   def guarded_sql(query: str) -> str:
       if any(tok in query.lower() for tok in DENY):
           return "BLOCKED by policy: destructive statement."
       ...
   

   With the Anthropic tool runner, gate inside the tool function (return "user declined" instead of executing); in LangGraph, use an interrupt() before the risky node (pb-41).

3. 3.

   Layer 3 — output validation. Every structured output passes a Pydantic schema; every user-facing text output passes checks for leaked secrets (regex over key formats), PII (reuse pb-43's redactor), and claims-without-source if your spec requires citations. Failed validation retries once with the error message, then fails closed.

4. 4.

   Layer 4 — budgets. Three counters per run, checked in the loop: steps (e.g. 12 tool calls), tokens (accumulate usage.input_tokens + usage.output_tokens per turn), dollars (tokens × your model's price table). On breach: stop, summarize partial findings, emit a BUDGET_EXCEEDED status. Add a daily aggregate cap across all runs at the API-key level via your provider dashboard or gateway (pb-48) — the loop cap protects a run; the key cap protects the month.

5. 5.

   Layer 5 — kill switch + audit log. A flag the loop checks every iteration (file, Redis key, or env var) so a human can halt a live run; structured logs of every tool call (run id, tool, args hash, outcome, cumulative cost). The log is what turns "the agent did something weird" into a five-minute diagnosis.

6. 6.

   Red-team your own agent. Ten adversarial inputs: injection in retrieved text, requests for denied actions, impossible tasks (to trigger budgets), secret-extraction attempts. Record pass/fail per layer.

Tool variants

• Guardrails-style validator frameworks (Guardrails AI, provider moderation endpoints) for input/output screening instead of hand-rolled checks.
• Gateway-level enforcement (LiteLLM budgets/rate limits per key, pb-48) — the org-wide complement to in-loop budgets.
• LangGraph-nativebudgets as state fields checked in conditional edges; kill switch as a checkpoint-and-abort.

Verification

• •
  All ten red-team cases behave as designed, with the blocking layer identified for each.
• •
  A forced runaway (impossible task, budget set low) stops at the cap and emits the partial-findings summary.
• •
  Cost accounting matches provider-reported usage within a few percent over five runs.
• •
  The kill switch halts a live run in under one loop iteration.

Pitfalls

• Prompt-only guardrails. "Never delete data" in a system prompt fails exactly when it matters — under injection.
• Budgets without graceful degradation. A hard kill that discards 40 completed steps wastes the spend; always summarize partial work on breach.
• Deny-list string matching as the only SQL defense.SELECT ... INTO, comment tricks, and encodings slip past naive matching — pair with a read-only database role (pb-43).
• Logging args in plaintext. Tool args can contain sensitive data; hash or redact them in the audit log.

Restricted-environment variant

Every layer here is provider-independent and runs against self-hosted models unchanged — budgets count tokens from the vLLM endpoint's usage fields; validators and deny-lists are local code. In client environments, set the dollar budget in the client's currency of concern (rows read, records exposed) as well as tokens: data-exposure budgets matter more than API spend when the model is free and local.

Success criteria

• •
  Five layers implemented on a real agent, each demonstrably blocking its class of failure.
• •
  Red-team results table: ten cases, expected vs actual, layer credited.
• •
  Per-run and per-day cost caps live, with measured accounting accuracy.
• •
  A one-page "agent safety spec" a reviewer could audit the implementation against.