When to judge, and when judging is malpractice

Entitlement lab: Module 4's three binary checks decide all 63 graded rows, catch all 47 failures and raise 0 false alarms, so a model judge adds no verdict anywhere. What it can add is a quotation. Three checks make 8 signatures, the corpus fills 5, and the one cell holding 26 failures spans 6 mechanisms — while 8 distinct span-detail strings already name 8 of the 9 mechanisms over 44 cases. One mechanism has no failing span at all: 3 rows of 63, 4.8% of the set. The cascade forwards 19 rows to the judge and 16 were already cleared, so 16 of 19 calls and 17,600.0 ms of a 20,900.0 ms step — 84.2% — buy no verdict. Pointwise makes 63 calls where pairwise needs 752, and one prompt over every criterion on every row costs 189, which is 63.0x the residue.

This Evals Engineering lesson is part of a paid plan. Sign in with a plan that includes this course to read it.