LASR-Callum/qwen3.6-27b-lora-500k-da20-numina

Qwen3.6-27B — 500k, 20% difficult-advice + maths-weighted remainder

LoRA adapter trained on 499,344 tokens with loss on assistant tokens only, empty-think markers excluded from the loss, for 1 epoch at lr 4e-5.

Training data: qwen3.6-27b-mixture-500k-da20-numina.

Source	Examples	Tokens	Share	Think block
difficult-advice (synthdoc_v2)	56	95,813	19.19%	real reasoning trace
NuminaMath-CoT	492	269,451	53.96%	none
No Robots	215	67,345	13.49%	empty marker
TULU3	94	66,735	13.36%	empty marker
Total	857	499,344

Within the non-difficult-advice 80.8%: NuminaMath 66.8%, TULU3 + No Robots 33.2%.

Trait balance

The difficult-advice half holds exactly 7 examples for each of the constitution's 8 principles. That quantisation is why its share lands at 19.19% rather than exactly 20%: 7 per trait gives 95,813 tokens, 8 gives 109,109 (21.8%). Exact trait balance was preferred over an exact 20%.

Think-block convention

Three different treatments, one per data type:

Data	Renders as	In the loss?
difficult-advice	<think>real reasoning</think>	yes -- this is the signal
TULU3, No Robots	<think>\n\n</think>	no -- context only
NuminaMath-CoT	no block; its CoT is in the response text	n/a

The empty marker is Qwen3.6's non-thinking marker, placed exactly where apply_chat_template puts it. It is masked from the loss: the model is conditioned on it but never trained to emit one, since learning to emit an empty think block is the documented reasoning-collapse pattern. NuminaMath is left unmarked because marking it "non-thinking" would contradict its own chain-of-thought content.

Training

Supervised	393,356 / 499,344 = 78.8%
Epochs / steps	1 / 54
lr / schedule	4e-5, cosine, 3% warmup
Runtime	35 min, 1x H100 80GB
r / alpha / dropout	32 / 64 / 0.05
batch x grad-accum	1 x 16
max seq len / packing	3072 / off
Final loss	0.946
Token accuracy	0.776

Verified before training, on the box: zero empty-think markers inside any supervised span, zero user or system tokens in the loss, and all 56 difficult-advice rows retaining their real reasoning traces.

loss_type: nll is set because TRL's default chunked-CE path patches the LM head and reads forward.__func__, which fails on this checkpoint's functools.partial forward. The loss is mathematically the same.

Related runs

Run	Data	lr	Loss
500k maths-weighted	no difficult-advice	1e-4	0.897
500k maths + empty-think	no difficult-advice	4e-5	0.878
this	20% difficult-advice	4e-5	0.946

Loss rises with the difficult-advice share because open-ended advice with reasoning traces is a harder next-token target than mathematical solutions. That is a property of the data, not of model quality.

Not yet evaluated on ODCV-Bench or agentic-misalignment.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-lora-500k-da20-numina")
model = model.merge_and_unload()

Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language checkpoint.