The demo worked; that proves almost nothing. This playbook walks a POC through Well-Architected lens question sets with AI gathering evidence — producing a sized gap register across security, reliability, cost, evals, and operations that tells the business what production actually costs.
Tier: Core · Hands-on time: ~30 minutes
The industry context is brutal: Pluralsight's 2026 research puts the share of organizations seeing zero GenAI ROI at roughly 95% — a statistic worth citing with its source, and one whose mechanism is visible from inside any engagement: POCs get funded, demos impress, and the unglamorous distance between demo and dependable production system never gets sized, so it never gets budgeted. The hyperscaler lens question sets (AWS Well-Architected GenAI lens and siblings) make that distance checkable — structured questions an AI can walk efficiently while the architect validates evidence and owns every risk rating.
Establish what the POC actually demonstrates. Interview the builders for the honesty list: hardcoded values, cherry-picked demo inputs, skipped auth, the happy path's boundaries. This list seeds the assessment — every faked element is a gap by definition, and surfacing it now is cheaper than an assessor discovering it.
Walk the lens question sets with AI as evidence-gatherer. Vikram, assessing a BFSI document-copilot POC before his ARB will hear the production pitch:
Walk this POC through a production-readiness assessment structured by
the Well-Architected GenAI lens question areas (attached: lens question
list, POC repo tree, architecture sketch, builder honesty-list).
For each question: (a) what evidence exists in the attached material —
quote it; (b) what evidence is ABSENT — absence is a finding, not a
pass; (c) draft gap statement where the POC falls short of a
production answer.
Rules: do not rate severity — I rate. Do not infer capabilities the
artifacts don't show; if the repo doesn't demonstrate it, it doesn't
exist. Cover at minimum: authn/authz, tenancy, data handling and
retention, eval coverage, failure behavior under 429/timeout, cost
instrumentation, rollback path, and operational ownership.
The AI drafts findings with quoted evidence; you validate each quote against the artifact before it survives.
Rate and reject findings yourself. Walk every drafted gap: confirm real ones, kill hallucinated ones (AI assessors invent both problems and reassurances), and rate severity by production consequence. The rejected-findings list stays in the record — it documents assessment rigor.
Spot-check the quality claims. If the POC claims answer quality, run a 20-item golden sample through the pb-33-style harness now. POC quality measured on the builder's favorite questions is the single most common inflated claim in these reviews.
Size the gaps in engineering terms. Group the register by theme (security, reliability, evals, cost, ops) and attach a T-shirt size and dependency note to each — "document-level authz: L, requires ACL sync design (pb-31)." The sizing turns the register from a critique into a plan the sponsor can fund.
Write the two-audience verdict. Engineering gets the full register. The sponsor gets one page: what the POC proved, what production requires (sized), the recommended path — graduate with funded gap closure, extend the POC to de-risk specific gaps, or stop — and the ROI-mechanism sentence: unfunded gap closure is how AI projects join the zero-ROI statistic.
Convert the register into the production design's backlog. Each accepted gap maps to a design activity in this track (tenancy → pb-31, HA → pb-44, evals → pb-33/46, cost → pb-42/47, approvals → pb-38) — the register is the index into the real architecture work.
POC assessments routinely touch client code and data-handling evidence — MSA constraints first, as ever. Run the lens walkthrough in client-approved tenancy, or sanitize: the lens questions themselves are public, and a structurally-faithful description of the POC (component types, auth posture, data classes — no client names or code) supports a useful external-assistant pass. Quality spot-checks run against client-tenancy endpoints with client data never leaving the boundary; a local Ollama-driven harness covers the strictest cases. The gap register is client work product and lives in their systems — and in services engagements it is often the artifact that wins the production phase, so its evidence discipline is commercial hygiene too.
The lens questions make the POC-to-production distance checkable — AI walks them and drafts evidence-quoted findings, you validate quotes and own ratings, and the sized gap register is what stands between the sponsor and the zero-ROI statistic.