Apply Your Prioritization Rubric Consistently with AI

Workflow area: Strategy & Roadmap · Tier: Foundation · Hands-on time: ~30 minutes

Prioritization frameworks fail in practice for a boring reason: humans apply them inconsistently. The item scored at 9 a.m. gets a rigorous Reach estimate; the item scored after lunch gets a vibe. A model applies the same rubric with the same energy to item 1 and item 50 — but it has no idea what your CEO promised a customer last week. The division of labor is the playbook.

When to use — and when not to

• Use it for batch scoringquarterly planning, a backlog groom of 30+ candidates, or merging asks from five stakeholder lists into one ranked view.
• Use it to expose rubric ambiguitywhen the AI scores something "wrong," half the time your rubric was underspecified; that's a fixable defect.
• Use it for the first pass, never the final callthe output is a draft ranking to react to, which is faster than a blank spreadsheet.
• Skip it for the strategic fewthe three bets that define your year deserve real debate, not a scoring shortcut; frameworks inform those calls, they don't make them.
• Skip it when the rubric itself is contestedif the team doesn't agree what Impact means, AI will just apply the disagreement consistently.

The recipe

1. 1.

   Write the rubric down sharply enough for a stranger. Each factor with its scale and anchored examples: "Reach = users touched per quarter; 3 = >50% of WAU, 2 = 10-50%, 1 = <10%." If you can't anchor a factor with examples, the humans were never scoring it consistently either.

2. 2.

   Prepare the candidate list. One line per item: ID, title, one-sentence description, and any evidence links (theme from pb-02, revenue at stake, ticket counts). Garbage descriptions produce garbage scores — the model scores what you wrote, not what you meant.

3. 3.

   Run the scoring pass. In ChatGPT or Claude:

   Copy

   Apply this prioritization rubric to every candidate below.
   
   Rubric:
   [PASTE FULL RUBRIC WITH SCALES AND ANCHORED EXAMPLES]
   
   For each candidate output one row:
   ID | R | I | C | E | Total | one-line rationale per factor score
   
   Rules:
   - Score ONLY from the information given. If a candidate lacks the
     information a factor needs, score it "?" and say what's missing —
     do not guess.
   - Apply the anchors literally; when torn between two scores, take
     the lower and say why.
   - After the table, list: (a) candidates you found ambiguous,
     (b) any rubric ambiguity that made scoring hard.
   
   Candidates:
   [PASTE CANDIDATE LIST]
   

4. 4.

   Review the "?" items and the ambiguity list first. Good output looks like:

   Copy

   C-17 | R:3 | I:? | C:2 | E:1 | — | Impact unclear: description says
   "improves onboarding" with no metric or evidence attached.
   
   Rubric ambiguity: "Confidence" anchors reference user evidence, but
   several candidates cite only internal stakeholder requests — unclear
   whether those count as evidence.
   

   Every "?" is a candidate whose write-up needs work, and every ambiguity flag is a rubric fix — this diagnostic output is worth as much as the scores.

5. 5.

   Override with reasons, in writing. Walk the ranked list and change what your judgment says to change — contractual commitment, strategic bet, a dependency the rubric can't see. Record each override in an override column: new score, reason, your initials. Ten overrides out of fifty is a healthy rubric; thirty overrides means fix the rubric, not the scores.

6. 6.

   Spot-audit before the list ships. Pick five scored items at random and re-score them yourself against the anchors. If you land within one point of the model on all five, trust the batch; if not, tighten the anchors and re-run.

Make it yours

• Google Sheets round-trip: ask for the output as CSV, paste into your existing RICE sheet, and keep AI scores, human overrides, and reasons in adjacent columns — the diff between the two columns is your judgment, made visible for the planning meeting.
• Productboard or Jira context: export candidates with their linked insights so the model scores against attached evidence counts, not just titles; re-import the final ranked list once overrides are recorded.
• Claude vs ChatGPT consistency check: for a high-stakes cycle, run the same rubric and candidates through both models. Items where the two disagree by more than two points are exactly the ambiguous ones worth human debate time.

Quality check

• Scores trace to anchorsread the rationale on five random items; "seems important" is not an anchor match, and a rationale citing evidence the candidate doesn't mention is a hallucination.
• The "?" list is non-empty on a real backloga model that confidently scored every under-specified item was guessing; that's a red flag, not a convenience.
• Effort scores went to engineeringAI estimates of engineering effort are folklore; the E column isn't final until someone who ships code has seen it.
• Every override has a written reasonan unexplained override is invisible politics; the override log is what makes the final ranking defensible.
• The math checksrecompute Total for a few rows; models drop factors and botch division in RICE arithmetic more often than you'd think.