AI features are priced in tokens but managed in units the business understands: cost per request, per conversation, per tenant, per 100K tokens. This playbook builds the unit-economics model across hosted API, provisioned throughput, and self-hosted GPU — and designs the tagging that makes the numbers observable from day one.
Tier: Advanced · Hands-on time: ~40 minutes
The FinOps evidence says this discipline arrived fast and painfully: per the State of FinOps 2026 coverage, 98% of practitioners now manage AI spend (up from 63% the prior year), yet 73% of AI projects run over budget — and cost per 100K tokens has emerged as the core unit-economics metric, with 55–80% of enterprise AI GPU spend now going to inference per the same practitioner literature. The architect owns two things here: the model that predicts unit cost, and the design-time attribution (tagging, virtual keys, metering) without which unit cost can never be measured. Retrofitting attribution is far harder than designing it in — treat that as the working assumption.
Define the units before the math. Pick the unit the business runs on — per conversation, per document processed, per agent run, per tenant-month — and express everything in it alongside the practitioner-standard cost per 100K tokens. A model in units nobody budgets in convinces nobody.
Measure the token profile from reality. Run 50–100 representative requests through the prototype and record input tokens, output tokens, and their spread. Token counts are the model's foundation and the place intuition fails worst — long system prompts and tool schemas make input tokens dominate in most agentic workloads.
Structure the three-way model with AI, prices from sheets. Anita, modeling orbit-docs-copilot's economics:
Structure a unit-economics model for a RAG copilot. Business unit:
cost per resolved conversation. Measured profile: mean 6 turns, 3.1K
input / 420 output tokens per turn, 22% of input tokens are a stable
prefix (system prompt + tool schemas).
Arms: (a) hosted API pay-per-token, (b) provisioned throughput,
(c) self-hosted on GPU instances (include utilization as a variable —
idle GPU is the hidden cost).
For each arm: the cost formula in terms of named variables, the
break-even conditions between arms as volume grows, and which variables
dominate sensitivity. Add rows for embedding and reranking costs.
Leave every price as "VERIFY: <price sheet>" — I fill rates myself.
Do not estimate any actual currency value.
Model the levers as variables, not vibes. Cache hit rate on the stable prefix, model-tier mix (what share of requests a cheaper model could serve), and context-length control each get a column — this is where the pb-47 cost stack plugs into the economics. Show unit cost at pessimistic, expected, and optimistic lever settings.
Treat self-hosted honestly. The GPU arm's real driver is utilization: instance-hours are paid whether tokens flow or not, and the practitioner finding that inference now dominates enterprise AI GPU spend is a warning about scale, not an endorsement of self-hosting at low volume. Include engineering and operations cost as a named line, even if estimated coarsely — omitting it is the classic self-hosted-looks-cheap error.
Design attribution into the architecture. Every model call tagged with feature, tenant, and environment; gateway virtual keys per team or tenant; metering events for the business unit (a "conversation resolved" event that joins to token spend). Write this into the design doc as requirements the build team implements — this is the design-time work that makes next quarter's numbers exist.
Close the loop against actuals. Once traffic flows, compare gateway spend data to the model monthly; divergences are findings (token profile drifted, cache underperformed, tier mix shifted) that update either the model or the design. An unreconciled cost model decays into fiction within a quarter.
Volume data, tenant counts, and unit economics are commercially sensitive — for client engagements, check the MSA before workload numbers reach an external assistant, and abstract where needed ("SaaS support workload, order-of-magnitude 10⁵ conversations/month"). The measurement half is naturally safe: token profiling runs against client-tenancy endpoints (Bedrock or Azure OpenAI in the client subscription) or a local Ollama model with the target tokenizer for structure, with rates filled from public sheets. Where the deliverable is a client business case, the model workbook is client work product — build it in their approved environment, and keep the assistant's role to structure and critique on sanitized inputs.
Measure the token profile, model all three arms with sheet-verified rates and utilization honesty, and design the tagging that makes cost-per-unit observable from day one — the 73%-over-budget statistic is what skipping these steps looks like.