The eval dashboard is green. The LLM judge scores nearly every answer 8 or 9 out of 10, week after week. Then a user forwards a transcript where the bot confidently quoted a processing time that contradicts the policy document — and the judge had scored that exact answer 9/10. Spot-checking finds more: long, fluent, wrong answers scoring higher than short correct ones.
A team replaced slow human review with an LLM judge to grade answers at scale. The judge prompt asked for a 1-to-10 quality score with no evidence requirement, and nobody ever compared its verdicts to a human's on the same answers. Most common right after a team discovers LLM-as-judge and wires it in during the same afternoon. Linked module: 3.7 (Evaluating LLM systems).
The judge is an unvalidated model grading another model, and it fails in known ways. Verbosity bias: judges read longer, more confident answers as better, so extra sentences hide the wrong fact instead of exposing it. Leniency: on a vague "rate this 1-10" prompt, most judges cluster at the top of the scale, so 9/10 carries almost no information. Verdict-first output makes it worse — a judge asked for the score before any evidence rationalizes instead of checking. Confirm the diagnosis the only way possible: pull ten transcripts, score them yourself against the written rubric before looking at the judge's scores, then compare. Every disagreement is either a judge failure, a wrong human label, or a rubric line two readers can read differently — and all three need fixing.
Rewrite the judge prompt to a binary verdict on one criterion — "is every claim in the answer supported by the context, true or false" — with defined terms, and force an evidence field before the verdict field so the judge must quote its proof first. Add "do not reward length or confidence" explicitly. Then audit: hand-score ten to twenty cases, run the new judge on the same cases, and resolve every disagreement before the judge grades anything that matters. Include planted wrong answers in the audit set — a judge tested only on good outputs proves nothing. Treat unparseable judge output as a fail, not a pass.
A judge is a model you ship, so it gets an eval too: never let judge scores into a report before the judge has been audited against human labels on the same rubric. Re-audit whenever the judge prompt, the judge model, or the kind of input changes. Keep spot-checking a small sample of judge verdicts forever — agreement measured once is a snapshot, not a property. And never let a model be the sole judge of a comparison its own family is competing in.
Q: "You use an LLM to grade your LLM's answers. Why should I trust that?" Strong answer: name the biases without prompting — position, verbosity, self-preference, leniency — then describe the audit: hand-label a sample, measure agreement with the judge, resolve disagreements, and only then let it grade at scale, with ongoing spot checks. Bonus for "binary verdicts with evidence before the verdict" and for refusing 1-to-10 scales because they cannot be calibrated.