AI Product Fundamentals & Economics

The scenario — the architecture chosen by demo

Priya — the Chennai invoice-workflow PM from Module 3 — got her "add AI" mandate in Q2: a support copilot that answers customer questions about invoices, approvals, and policies. A vendor demo sealed the approach in one meeting. The vendor's engineer fine-tuned a model on a sample of the company's help articles, and the demo answers were fluent, on-brand, and specific. Leadership approved fine-tuning as the architecture, a quarter of engineering time, and a launch target.

The unraveling was slow and expensive. The help content changed weekly — every product release altered policies and screens — and each change meant the fine-tuned model was confidently teaching customers last month's product. Retraining runs became a recurring cost and a recurring delay. Meanwhile nobody had modeled per-query economics: at the projected 200K queries a month on a frontier-tier model, serving cost was on track to consume roughly a third of the feature's attributable revenue — before anyone got their support-ticket deflection numbers to hold up. Eight months in, an engineer prototyped the "boring" alternative in a week: retrieval over the live help center with a mid-tier model, no training runs, always current. It matched the fine-tune on their (belatedly written) eval set and cost a fraction per query.

The fine-tune wasn't a stupid idea. It was an unexamined one — chosen by demo impressiveness rather than by the four questions this module teaches: how volatile is the knowledge, what does quality actually require, what does a query cost at projected volume, and what does the P&L look like at your price point. Every one of those is a PM question. Every one has auditable math.

What You'll Learn

• The capability spikea PM-runnable, one-day prototype-plus-mini-eval that kills bad roadmap bets cheap
• RAG vs fine-tune vs prompt-only as a product decisionvolatility, cadence, cost curves, latency, data rights
• Model tiers and routing economicsthe quality-vs-cost frontier built from your eval set, not leaderboards
• Token unit economicscost per interaction, margin impact, and the gross-margin honesty conversation
• Pricing and packagingseat add-on, usage credits, hybrid, outcome-based, and bill-shock prevention
• Vendor bake-offs and platform decisionseval harnesses over demos; API vs platform vs self-host with India-aware rows

Before any of it — the one-day capability spike

The cheapest money you will ever save on an AI feature is the quarter you don't commit to one that can't work. Before a model-backed feature enters the roadmap, run a capability spike: one day, PM-runnable, no engineering sprint.

1. 1Collect 20 real cases. Actual inputs the feature would face — real support questions, real documents, real data. Not cherry-picked; grab the last twenty, ugly ones included.
2. 2Prototype the interaction in a workbench. Anthropic's Workbench, OpenAI's playground, or Google AI Studio: system prompt, sample context (paste the relevant help articles by hand — you are faking retrieval), run all twenty cases.
3. 3Grade with a mini-eval. Simple pass/fail per case against criteria you write down first. promptfoo runs this systematically if you want; a spreadsheet is fine on day one.
4. 4Decide with a number. Fourteen of twenty passing with visible fixes is a green light for deeper investment. Six of twenty with fundamental misses — needs data you can't provide, reasoning it can't do — kills the bet for the cost of a day.

The spike does not tell you the feature will succeed. It tells you whether quality is plausibly reachable — and it produces, as a side effect, your first twenty eval cases and a realistic sense of tokens per interaction, which the rest of this module's math needs.

RAG vs fine-tune vs prompt-only — a product decision, not a religious one

Three ways to get product-specific knowledge and behavior into an LLM feature, and the decision belongs to the PM because every input is a product fact, not an ML fact:

The heuristic that decides most real cases: volatile knowledge wants retrieval; stable behavior wants fine-tuning; and start prompt-only until proven otherwise. Priya's help content churned weekly — the volatility row alone rules out fine-tuning as the knowledge mechanism. Fine-tuning earns its place for stable behavioral shaping (a strict output format, a classification scheme, brand voice) — usually on a small model, usually later, and the 2026 consensus is blunt: most production LLM features never need it.

The decision memo (Lab 11's artifact) resolves the dimensions for your case, at your projected volume, with TCO math you personally audited — and includes revisit triggers: "we chose RAG; revisit if query volume passes X or content churn drops below Y." Model prices and capabilities shift quarterly; an architecture decision without revisit triggers is a future surprise with your name on it.

Model tiers and routing — the frontier is yours, not the leaderboard's

Every provider ships a tier ladder — frontier models for hard reasoning, mid-tier workhorses at a fraction of the price, small models cheaper still — with per-token prices that differ by an order of magnitude or more between rungs. The PM decision: which rung does each feature need?

The wrong way to answer is public leaderboards, which measure aggregate benchmark performance, not your task. The right way is the muscle you build in Module 7: run your eval set across tiers and plot the quality-vs-cost frontier. The recurring, budget-changing discovery is that a mid-tier model matches frontier quality on your task — summarizing invoices and answering policy questions is not competition math — at a tenth the cost. The equally important trap (Lab 12 plants it): a cheap model that wins headline accuracy while failing your two highest-severity failure modes. Averages hide severity; your eval set weights what your product actually cannot afford to get wrong. That is why the frontier must be yours.

Routing is the production version of this insight: default to the cheap tier and escalate the hard cases — via a classifier, confidence signals, or explicit user action ("thinking harder…"). Reported cost reductions from routing at volume are large (vendor-published 70–90% figures circulate; treat them as directional, not promised), but the PM-owned part is the escalation contract: which failures are acceptable from the cheap path, what triggers escalation, and what the user experiences in between. Tools like OpenRouter and LiteLLM make multi-model bake-offs and routing pilots one-key cheap.

Token unit economics — the spreadsheet leadership hasn't seen

The unit-economics model is four lines of multiplication, and building it is the fastest credibility move an aspiring AI PM can make, because most teams ship first and compute later:

Cost per interaction = (input tokens × input price) + (output tokens × output price), summed across every call in the interaction — retrieval-augmented context, the drafting call, the safety check, the retry rate. Real interactions are rarely one call.

Monthly serving cost = cost per interaction × interactions per active user × active users. Then the two honesty steps:

• Sensitivity, both directionsmodel prices have fallen repeatedly, but your usage grows too, and a provider repricing or deprecating a tier can move your COGS 3× in a quarter. Model the range, not the point estimate. Surveys consistently find most IT leaders hit unexpected AI consumption charges; the surprise is the norm you're engineering against.
• The gross-margin conversationclassic SaaS runs 80–90% gross margins; AI-heavy features run 50–60% because every use has real marginal cost. That's a different business, and someone has to say it out loud before the pricing meeting — with a spreadsheet, calmly, while the decision is still cheap. That someone is you.

The India margin scenario

The math bites hardest where ARPU is in rupees and COGS is in dollars. A concrete illustration of the shape (round numbers, for the mechanics):

An India-market SaaS plan at ₹599/month (~$7). The AI feature costs $0.012 per interaction on a mid-tier model, and an engaged user runs 150 interactions a month: $1.80 ≈ ₹150 — a quarter of the entire plan price, before you've paid for anything else. The same feature inside a $99 US-market plan is under 2% of revenue — a rounding error. Identical feature, identical model, radically different business.

Consequences for India-market pricing: heavier-usage features need caps, credits, or a paid tier rather than all-you-can-eat bundling; routing and small models are a margin necessity, not an optimization; and a weakening rupee moves your COGS while your price stays fixed. Any AI-feature P&L for an India product carries an FX-sensitivity row.

Pricing and packaging — four models and a bill-shock rule

More than half of AI-era SaaS pricing has shifted toward usage-based components, and the packaging choice is a PM decision with an economics dependency:

Whatever the model, bill-shock prevention is a product requirement, not a billing detail: visible usage meters, threshold alerts, soft caps with grace, and a designed what-happens-at-the-limit experience. Every horror story about a surprise five-figure AI invoice becomes churn, a negotiation, or a viral post — and the prevention is a week of product work.

Vendors and platforms — evidence, not demos

Two more decisions complete the economics kit, both governed by the same rule — production evidence, not demo impressions:

• The vendor/model bake-offrun your identical eval set across every candidate (promptfoo makes this mechanical; pre-generated outputs make it free), and write the decision memo from measured quality, cost per interaction, and latency. Demos are adversarially selected by definition; your eval set is the only demo you should trust. This is Priya's counterfactual — the week-one test that would have saved two quarters.
• Platform vs API vs self-hostdirect provider APIs (fastest capability access), cloud platforms like Bedrock/Vertex/Azure (procurement, compliance, and data-residency wrappers), gateways like LiteLLM (portability and spend tracking), or self-hosted open-weights (maximum control, real ops burden). The India-relevant rows: data residency (client contracts and DPDP posture often decide this before anything technical does — check which providers offer in-region processing), model deprecation risk (providers retire models; a deprecation calendar is part of the decision), and gateway lock-in vs. leverage (a thin routing layer preserves your ability to re-decide everything else). Write the choice as a brief with revisit triggers, like every other decision in this module.

Try it now

1. 1Spike one bet. Pick an AI feature idea from your roadmap (or invent the support copilot). Collect 20 real cases, run them in a workbench with hand-pasted context, grade pass/fail against criteria you wrote first, and issue a green/red verdict with the number.
2. 2Build the four-line model. Estimate tokens per interaction from your spike, multiply out to monthly cost at three volume scenarios using current provider prices, and compute the feature's gross margin at your product's actual price point — India ARPU if that's your market.
3. 3Write the architecture paragraph. For the same feature, answer in one paragraph each: knowledge volatility, quality requirement, cost at volume, data rights — and conclude prompt-only / RAG / fine-tune with two revisit triggers. That paragraph is the interview answer.