Sanity-Check Every Big Estimate with the Outside View

Workflow area: Delivery & Reporting · Tier: Foundation · Hands-on time: ~25 minutes

The planning fallacy is not a junior mistake — it is how human estimation works. The inside view (decompose the task, sum the parts) systematically omits the unknown unknowns, the integration pain, and the three-week wait for another team, because none of those appear in the decomposition. The corrective is reference-class forecasting: ask what things like this took, not what this thing should take. AI is a genuinely good partner for this — not because it knows your codebase, but because it is a tireless generator of the reference questions, risk categories, and doubling scenarios that inside-view thinking suppresses. Your tracker history supplies the data; the model supplies the interrogation.

When to use — and when not to

• Use it before committing any estimate upwardonce a number leaves your mouth in a planning meeting, it anchors everything; sanity-check it while it is still private.
• Use it on estimates that feel comfortablecomfort is the symptom; an estimate nobody winces at has usually not met its risks yet.
• Use it when reviewing a team's estimate you suspect is hopefulthe reference-class questions let you challenge the number without challenging the people.
• Skip it for small, reversible worka two-day task estimated wrong costs two more days; ceremony here erodes the habit for when it matters.
• Skip treating the AI's numbers as datathe model's "typically takes 3-6 months" is pattern-matched prose, not evidence; your own tracker history is the only reference-class data that counts, and where you have none, the output is questions to investigate, not answers.
• Skip using it to bludgeon estimates downwardthis playbook only ever finds reasons estimates are too LOW; if you catch yourself using it to argue a team's number down, you are holding it backwards.

The recipe

1. 1.

   Write down the estimate and its basis. The number, who made it, and the one-paragraph inside view: the plan's steps and what each is believed to cost. Also pull your reference data: similar past projects from the tracker — what was estimated, what was delivered, and the ratio. Three examples beat zero; if you truly have none, note that this estimate is uncalibrated and proceed anyway for the risk questions.

2. 2.

   Run the outside-view interrogation.

   Copy

   Stress-test a software estimate using reference-class thinking.
   Do not soften. Your job is to find why this estimate is too
   low, not to validate it.
   
   THE WORK: [ONE PARAGRAPH — what is being built/changed]
   THE ESTIMATE: [e.g. "6 weeks, 3 engineers"] made by [team /
   individual / me]
   OUR REFERENCE DATA: [PASTE — e.g. "similar migration in 2024:
   est 8 wks, took 15. API rebuild: est 6 wks, took 9. No third
   example."]
   
   Produce:
   1. REFERENCE-CLASS QUESTIONS — 5 questions that establish what
      class of work this really belongs to (not what makes it
      special — what makes it typical), each answerable from our
      tracker or memory.
   2. OUR OWN BASE RATE — from my reference data only: the
      estimate-to-actual ratio, applied to this estimate. Show
      the arithmetic. If data is too thin, say "insufficient — 
      treat estimate as uncalibrated" rather than inventing an
      industry number.
   3. OMITTED-RISK SCAN — go category by category: integration
      with existing systems, cross-team dependencies and their
      wait times, migration of existing data/users, review and
      compliance gates, environment/tooling setup, testing beyond
      happy path, the estimator's availability vs. assumption.
      For each: likely omitted here, or visibly covered?
   4. THE 2X QUESTION — the three most plausible specific ways
      this becomes double the estimate, written as concrete
      scenarios ("the partner sandbox takes 3 weeks to
      provision"), not categories.
   5. CHEAP TESTS — for the top 2 risks: what could be checked
      this week, for less than a day's effort, that would
      collapse the uncertainty?
   

3. 3.

   Take the questions to reality. Good output looks like:

   Copy

   OUR BASE RATE: two data points, ratios 1.9x and 1.5x. Applied:
   6 weeks → 9-11.5 weeks. Two points is thin — treat as a
   directional warning, not a forecast.
   OMITTED-RISK SCAN:
   - Cross-team wait: plan assumes platform team reviews the
     schema "when ready" — their current review queue is 2 weeks.
     LIKELY OMITTED.
   THE 2X QUESTION:
   - Legacy data has undocumented formats; migration scripts hit
     edge cases weekly for a month.
   CHEAP TESTS:
   - Sample 500 legacy records for format anomalies — half a day,
     kills or confirms the biggest 2x scenario.
   

   Answer the reference-class questions from your tracker. Run the cheap tests — they are the highest-leverage item in the whole playbook: a half-day probe that converts "might double" into "measured" changes the estimate more than any discussion.

4. 4.

   Reconcile with the team, inside view meets outside view. Bring the base-rate math and the omitted risks to the estimators as questions, not verdicts: "our last two similar projects ran 1.5–1.9x — what makes this one different, specifically?" Sometimes there is a real answer (smaller scope, known codebase, done twice before) — the outside view is a prior, not a prophecy. The output is a revised estimate as a range, plus the risk list attached to it.

5. 5.

   Log the estimate for future calibration. Estimate, revised range, actual outcome when it lands — three columns in a running sheet. Six months of this and step 2's reference data stops being thin; your team's real base rate is the most valuable planning number you will ever own.

Make it yours

• Solo leader, no historical data: start the calibration log today and lean on the omitted-risk scan and 2X questions meanwhile — the risk interrogation needs no history to be useful; the base rate arrives with time.
• Estimation-mature org with story-point history: feed the model real distributions ("epics estimated at 20-30 points: median actual 1.6x, p90 2.4x") and ask for the p50 and p90 forecast of this estimate — then quote the p90 to anyone who needs the date to be safe.
• Reviewing upward estimates (you're the skeptical exec): run only steps 2-3 on the submitted estimate and bring the CHEAP TESTS to the review meeting — "before we commit, these two half-day probes would tell us which world we're in" is the most constructive challenge available.

Quality check

• The base rate came from your data, not the model's proseany "industry average" the model offered without your input is decoration; strike it.
• Reference questions got answered from the trackeran outside view built on remembered projects inherits the same optimism it was meant to correct; memory says 8 weeks, the tracker says 15.
• The cheap tests actually rancount them; an unrun cheap test on a 2x risk means the estimate shipped with a known, checkable blind spot.
• The team's "what makes this different" got a specific answer or the range widened"we'll be careful this time" is not a difference; a genuinely smaller scope is.
• The final number is a range with risks attachedif what got committed upward is a single number with the risk list stripped, pb-11's scenario discipline was undone at the last step.