Score Hanzi and MECO at every checkpoint
#1
by Raj-Sanjay-Shah - opened
Summary
- validate revision-labelled fast_eval_hidden payloads
- reject participant-supplied Hanzi and MECO checkpoint scores
- score hidden tasks server-side at every checkpoint
- merge those scores into fast_eval_results
- preserve multilingual fast results in evaluation output
- document checkpoint-level hidden-task scoring
Why
The evaluation pipeline now produces raw Hanzi and MECO predictions for each checkpoint. The Space must validate and score them using the same protected references as the final checkpoint.
Validation
- all 9 Space unit tests pass
- Python compilation checks
- git diff --check