# Claim 5: response length reduction


---
<!-- trackio-cell
{"type": "markdown", "id": "cell_3abf7a415447", "created_at": "2026-07-16T18:28:57+00:00", "title": "Method summary for this claim"}
-->
## Method summary for this claim

Average response length (in tokens, via the eval tokenizer) of XRPO-full's final sampled completions vs. GRPO-baseline's, on GSM8K-test and MATH-500 subsets — analog of the paper's Figure 4a. Response length reduction alongside accuracy gains would indicate XRPO learns to reason more efficiently rather than just more verbosely/more often correct.


---
<!-- trackio-cell
{"type": "markdown", "id": "cell_eaa98c603bea", "created_at": "2026-07-17T04:35:59+00:00", "title": "Result: Claim 5 NOT reproduced — response length increased, not decreased"}
-->
## Result: Claim 5 NOT reproduced — response length increased, not decreased

| config | gsm8k avg len (tokens) | math500 avg len (tokens) |
|---|---|---|
| grpo (baseline) | 361.0 | 372.6 |
| xrpo_full | 382.5 | 384.0 |
| xrpo_no_icl | 382.6 | 384.0 |
| xrpo_no_novelty | 382.5 | 384.0 |

Every XRPO variant produced **longer**, not shorter, responses than GRPO — the opposite of the paper's claimed 13.6%/13.4% reduction on AIME'24/'25. The gap (361 → ~383, a ~6% *increase*) is largely explained by `max_completion_length=384`: GRPO's completions terminate somewhat before the cap (allowing natural EOS), while every ICL/novelty-flagged variant runs almost entirely up against the cap (`completions/clipped_ratio` was frequently 0.9-1.0 in training logs for the XRPO variants vs lower for plain GRPO) — i.e. XRPO variants here more often fail to produce a terminated answer within budget, not that they reason more efficiently. This is consistent with a model that hasn't been trained long enough to have learned when to stop, combined with ICL-seeded prompts being longer/more complex (retrieved exemplars prepended to the question), which likely increases the reasoning trace length needed. This does not support Claim 5 at this reduced scale.