AI-Accelerated EDA: Hypotheses to Validated Findings Memo

Tier: Foundation · Workflow area: Exploratory analysis

Done right, you explore three times as many hypotheses in the same time; done lazily, you ship a memo full of confident artifacts. The 2025 Stack Overflow survey's top AI frustration — "almost right" code — is precisely the risk here, because almost-right EDA code still produces a chart.

When to use

• •
  Starting analysis on a dataset where you have domain questions but no fixed hypothesis list.
• •
  A stakeholder asks an open question ("why did Q2 conversion dip?") requiring systematic slicing.
• •
  You need a findings memo others will act on, not just personal exploration.

Skip the full loop for a single known cut ("plot revenue by month") — just write it.

Prerequisites

• •
  Jupyter/notebook environment with pandas, matplotlib or plotly; the dataset loaded and already profiled (see pb-01 — EDA on unprofiled data wastes the loop on quality bugs).
• •
  An AI assistant in or beside the notebook (Claude, ChatGPT, Cursor, Databricks Assistant).
• •
  A one-paragraph business context you can paste: what the data is, the decision at stake, known caveats.

The workflow

1. 1.

   Hypothesis generation, breadth-first. Paste business context plus schema:

   Copy

   Context: subscription SaaS, weekly signups table (2023-2026), columns:
   signup_date, channel, plan, country, trial_converted, mrr_first_invoice.
   Question: trial-to-paid conversion fell from 18% to 14% between March and
   June 2026. Generate 12 distinct hypotheses that could explain this, tagged
   [mix-shift | behavior-change | data-artifact | seasonality]. For each, state
   the exact cut of the data that would confirm or refute it.
   

   Insist on the data-artifact tag — instrumentation changes explain a large share of real "trends".

2. 2.

   Prioritize yourself. Pick 4–6 hypotheses by plausibility and cost to test. This judgment is your job; models rank by genericness, not by what you know about the March pricing change.

3. 3.

   Generate analysis code per hypothesis, with a falsification framing:

   Copy

   Hypothesis H3: the dip is a mix shift toward paid-social signups, which
   always convert worse. Write pandas code that:
   - computes conversion by channel by month,
   - decomposes the overall change into within-channel change vs mix change,
   - prints both components as percentage points.
   Aim to REFUTE the hypothesis; note what result would falsify it.
   

4. 4.

   Run, inspect, and challenge. Execute each cell. For anything surprising, ask: This shows within-channel conversion flat but mix explaining 3.2pp. List three ways this decomposition could be misleading given the schema above. (Simpson's-style aggregation, denominator shifts from a signup-form change, censoring of recent trials that have not finished their window.)

5. 5.

   Chase the artifact angle explicitly. Before believing any behavioral finding, test the boring explanation: definition changes, tracking gaps, timezone shifts, incomplete recent periods. trial_converted for June cohorts may simply not be mature — recompute with a fixed 30-day window.

6. 6.

   Write the memo with numbers pinned. Paste your verified tables and ask for a memo draft: findings ranked by confidence, each with the supporting number and the cut it came from, plus a "what we ruled out" section. Rewrite the interpretation sentences yourself — you sign the memo.

Tool variants

• Agentic notebook assistant (Claude Code, Cursor, Databricks Assistant): it can run cuts itself; review every executed cell and require it to show intermediate tables, not just conclusions.
• Chat assistant + manual notebook: copy code across; slower but forces you to read everything — a good default while calibrating trust.
• SQL-first (warehouse copilot): same loop with CTEs per hypothesis; keep a scratch schema for materialized cuts so results are re-queryable.

Verification

• •
  Every memo finding maps to a specific executed cell/query you can rerun.
• •
  The mix-vs-rate decomposition (or equivalent) reconciles: components sum to the observed total change.
• •
  Recent-period censoring checked for any metric with a maturation window.
• •
  At least one initially promising hypothesis was killed and appears under "ruled out" — a memo with zero rejected hypotheses means the falsification pass did not happen.

Pitfalls

• Narrative gravity. Once the model states a story ("paid social diluted quality"), later outputs bend toward it. Test hypotheses in separate conversation threads, or explicitly instruct "argue against the current leading explanation."
• Chart-driven confirmation. A generated chart of noisy weekly data "shows" whatever the title says. Demand the underlying table and effect sizes with denominators.
• Silent filters. Generated code often drops NaNs or filters "test" users without telling you. Diff len(df) before and after every transform.
• Overrunning the question. Twelve hypotheses is for exploration; the memo answers the stakeholder's one question. Cut ruthlessly.

Restricted-environment variant

• •
  Share schema and aggregate cuts only; run all code locally against raw data. Pasting aggregated tables (counts, rates by segment) into a hosted assistant is usually acceptable where row-level data is not — confirm with your data-governance policy.
• •
  Fully offline: use a local model (Ollama, 8B–70B class) for hypothesis generation and memo drafting — both are language tasks that degrade gracefully — and write the pandas yourself.
• •
  Strip customer identifiers and any small-cell segments (n < 20) from anything pasted; small cells can re-identify.

Success criteria

• •
  4+ hypotheses tested with executed code in a session where you would previously have managed 2.
• •
  Memo delivered with confidence-ranked findings, pinned numbers, and a ruled-out section.
• •
  A skeptical colleague picking any finding can be walked to the exact cell that supports it.
• •
  At least one data-artifact explanation was explicitly tested before any behavioral story shipped.