---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
tags:
  - chess
  - rl
  - dpo
  - distillation
  - qwen3.5
datasets:
  - Chess-Nut-Engine/chess-sft-corpus-4x
---

# chess-qwen35-0.8b-rl-20260710 — RL campaign checkpoints

Four checkpoints from the RL phase (2026-07-09/10), all descending from the
v3 SFT curriculum (`...sft-v3-20260709`). Gameplay scores are vs
Stockfish limited to Elo 1320, 1,024 games, thinking mode unless noted.

| subfolder | what it is | game score |
| --- | --- | --- |
| `topup` | v3-C + prompt-unified seed-pack top-up (the RL seed) | 0.1160 think / 0.1201 gut |
| `dpo_r2` | + iterative DPO on 30.8k stratified self-play pairs | **0.1337** think |
| `r5e_english` | dpo_r2 + English-voice SFT top-up (verified 35B traces) — **reasons in natural English (399/400 game traces)**, best diversity | 0.1260 think (statistical tie w/ dpo_r2) |
| `dopd_r1` | r5e + 200 steps DOPD token-routed distillation (arXiv 2606.30626 adaptation, same-weights privileged teacher) | pilot — see repo docs |

Key findings encoded in these checkpoints: thinking-mode play only pays when
trained-for (+0.018 for dpo_r2, nothing for the seed); preference training
nudges move choice but cannot shift trace style; one SFT top-up flipped the
model to natural-English reasoning at zero strength cost. Full records:
`docs/experiments/2026-07-10_rl_campaign/` in the project repo.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained(
    "Chess-Nut-Engine/chess-qwen35-0.8b-rl-20260710",
    subfolder="r5e_english", trust_remote_code=True)
t = AutoTokenizer.from_pretrained(
    "Chess-Nut-Engine/chess-qwen35-0.8b-rl-20260710",
    subfolder="r5e_english")
```

Planning protocol: `<think>...</think><move>uci</move>`; `<move>` tags are
tokenizer special tokens. Thinking mode requires the chat template's
`enable_thinking=True`.