Qwick-3.5-9B / DATA.md
horiuchinobuyuki's picture
Upload folder using huggingface_hub
9281e75 verified
|
Raw
History Blame Contribute Delete
1.87 kB

Data statement

This repository includes aggregate evaluation results. Training examples, benchmark prompts, answer keys, model responses, and third-party benchmark datasets are not distributed here.

Training data

The SFT stage selected 1,324 full-trace examples (630 English and 694 Japanese) from a 1,536-problem distillation split. The selected-data manifest SHA-256 is 623fa2f278c83d9c4a2423578fa4330d3d0aa252761663373a602cfa5a4ad150.

Step-DPO used 1,024 local preference pairs (768 strict and 256 ties). The preference-data SHA-256 is aa336a6dd6484d2a56b8c4570e32311965aa88b0f43ddc19a79e31e383beb3b1.

These hashes identify the training snapshots used for this checkpoint. They do not grant rights to redistribute the underlying records.

Evaluation data

Public evaluations cover MMLU-Pro, GPQA-Diamond, a 60-problem HMMT slice over four seeds, IFEval, a 987-item common JMMLU slice, a 1,055-item LiveCodeBench-v6-compatible manifest, and all 900 rows of the MMMU validation split. The MMMU result contains 30 rows per subject across all 30 subjects and is explicitly not the 10,500-row test split. Obtain those datasets from their publishers and follow their terms.

The reserved holdout contains 384 synthetic prompts: 192 English, 192 Japanese, and 128 each for choice, numeric, and exact-answer grading. It was separated from training and checkpoint search and evaluated once after selection under a separately frozen temperature-zero internal release gate. That gate passed; it is not presented as a public temperature-1.0 performance benchmark, and it was not used for retuning or rerun. heldout_results.json contains aggregate gate metrics and cryptographic identities; it does not contain prompts, answers, predictions, or traces.

Full generation traces were retained for evaluation and length analysis outside this model repository.