Slices of each corpus withheld from every training delivery and scored with the same verifiers, so the eval measures the work, not memorization. Refreshed as records arrive from operating sellers.
held-out eval · rcm twin · illustrative
figures shown are illustrative · real reports per delivery, refreshed on each record update
The held-out slice is carved before any training delivery is cut, per company and per task family, and its hashes are recorded so contamination is checkable rather than asserted.
Scored by the graders the environment ships with, against what the outside world said happened.
Where the original staff worked the tasks, their pass rate is reported beside the model's.
Pass rate, partial credit where the rubric allows it, and the distribution of failure modes, so a lab sees where a model breaks rather than a single number.
New held-out slices as new records arrive. A benchmark that was current a year ago is still current.
Held-out sets built to a lab's specification on a shared corpus, or on an exclusively licensed one.
Also in for labs
