Audit found three unrelated rectification scoring paths: the local-agent
methodology references, the production v5 server scorer, and the v4 fact
ranker that only the eval scripts import. The sealed holdout numbers
measure the third one, so neither the production path nor the local path
has ever been evaluated.
Task 0 is therefore a blind three-way bake-off on one case set, gating
every later task. The remaining tasks cover a single convergence function
with property tests, moving question authority fully to the server, making
the deliverable exist from turn zero, and the calibration data shortage.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0155nFCgCHtoA7jhSDGmZmMu
