Build a Sandboxed Data-Analysis Agent

Tier: Advanced · Workflow area: Agentic systems

The difference between a demo and something you can trust is twofold: the code runs in a sandbox that cannot touch production, and every headline number is re-computed once through an independent path before it ships.

When to use

• •
  Recurring ad-hoc analysis requests ("why did signups dip last week?") that each take you 30–60 minutes of pandas work.
• •
  Self-serve analytics for stakeholders who cannot write SQL, backed by datasets you control.
• •
  You want the agent pattern in your portfolio: plan → execute code → verify, with real guardrails.

Skip the agent for one-off analyses — driving an AI assistant interactively in a notebook is faster and safer. Agents pay off on the tenth repetition, not the first.

Prerequisites

• •
  An Anthropic API key (or equivalent) with a few dollars of budget; each solved task typically costs well under $1 on a mid-tier model.
• •
  A sandboxed Python executor: a Docker container with no network egress and a read-only mount of the dataset, or a hosted sandbox (E2B-style). Never exec() model-generated code in your host process.
• •
  A parquet/CSV extract of the data — never a live production connection for v1.

The workflow

1. 1.

   Build the sandbox first. A minimal Docker approach: image with pandas, matplotlib, duckdb; run with --network none, a read-only volume mount for /data, a writable /output, a 60-second timeout, and a memory cap. The executor function takes a code string, runs it, returns stdout/stderr and any files written to /output.

2. 2.

   Define the agent with one code tool. Using the Anthropic Python SDK's tool runner, the loop (call model → execute tool → feed result back → repeat) is handled for you:

   Copy

   import anthropic
   from anthropic import beta_tool
   
   client = anthropic.Anthropic()
   MAX_STEPS = 12
   steps = {"n": 0}
   
   @beta_tool
   def run_python(code: str) -> str:
       """Execute Python in the analysis sandbox and return stdout/stderr.
   
       Args:
           code: Python code. Data is at /data (read-only); write charts to /output.
       """
       steps["n"] += 1
       if steps["n"] > MAX_STEPS:
           return "STEP BUDGET EXHAUSTED. Summarize findings from results so far."
       return sandbox.run(code, timeout=60)  # your Docker/E2B executor
   
   runner = client.beta.messages.tool_runner(
       model="claude-opus-4-8",
       max_tokens=16000,
       tools=[run_python],
       messages=[{"role": "user", "content": TASK_PROMPT}],
   )
   for message in runner:
       pass  # last message is the final answer
   

3. 3.

   Write the system/task prompt as an analyst's contract. Require: state a plan before the first tool call; profile the data before analyzing it (dtypes, nulls, date ranges); every claimed number must come from executed code, not memory; end with a chart file path and a 5-sentence narrative.

4. 4.

   Add the independent verification step. After the agent finishes, send a fresh request (new conversation, same sandbox): "Re-compute only the headline number X using a different method than this transcript used, and report agreement or disagreement." Disagreement blocks delivery.

5. 5.

   Run it on three real questions you have previously answered manually. Compare the agent's numbers against your known-good answers before showing anyone else.

Tool variants

• Anthropic tool runner (shown above): least loop code to own; step budget enforced inside the tool function.
• Server-side code execution (Anthropic code-execution tool): Anthropic hosts the sandbox — declare {"type": "code_execution_20260120", "name": "code_execution"} in tools and upload the file via the Files API; you trade infra control for zero sandbox maintenance.
• LangGraph implementation: model the plan → code → critique loop as an explicit graph when you need checkpointing and resumability (see pb-41).

Verification

• •
  The verification pass re-computes the headline number via a different code path and agrees within tolerance.
• •
  Every number in the narrative appears in a tool-call output in the transcript — spot-check three.
• •
  The sandbox blocks a deliberate escape test: ask the agent to pip install a package and to read /etc/passwd; both must fail.
• •
  Step budget triggers correctly on an intentionally impossible question instead of looping forever.

Pitfalls

• Trusting the narrative over the transcript. Models sometimes state numbers that never appeared in any execution output. The transcript is ground truth; the prose is a claim.
• Live database credentials in the sandbox. One generated DROP or a runaway full-table scan and you own the incident. Extract to files first.
• Unbounded loops. A confused agent will happily burn 50 tool calls re-trying a broken approach. Budget steps and tokens from day one (see pb-44).
• Charts as verification. A plausible-looking chart of wrong numbers is worse than no chart. Verify the numbers, then trust the chart.

Restricted-environment variant

For client or regulated data: run the executor fully on-prem (local Docker), and swap the hosted model for a self-hosted one served by vLLM or Ollama with an OpenAI-compatible endpoint — the tool-calling loop is identical. Redact or pseudonymize column names and sample values in the prompt if even schema metadata is sensitive; the agent only needs the schema shape, not real customer identifiers.

Success criteria

• •
  Three previously-solved business questions answered correctly end-to-end without human edits.
• •
  Verification pass and escape tests both behave as designed, with evidence saved.
• •
  Cost per solved task measured and documented (typically cents to tens of cents at the data sizes that fit a sandbox).
• •
  A colleague can read the transcript and audit exactly where each number came from.