# Training and provenance

Grug 27B v2 starts from `Qwen/Qwen3.8-27B` revision `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`. It is a new post-training run from Qwen, rather than another adapter applied to the v1.1 checkpoint. The Qwen3.5-family architecture and original vision encoder are preserved.

## Main model

The training split contains 3,466 examples, with 176 examples in a task-group-disjoint validation split. The trainer evaluates a fixed 96-example subset of that validation split. Groups are split before reasoning-effort variants are expanded.

The mixture combines verified coding and math examples from training splits, verified repository repair episodes, inherited Grug agent/tool and general replay, and newly authored session-title examples. Generated code answers are tested before acceptance. Grug compression preserves final answers, tool calls, and fenced code. Long reasoning receives separate section compression to retain more of its original content. Raw inherited replay is not published.

Training uses rank-32 LoRA with alpha 64, learning rate 2e-5, microbatch 4, accumulation 2, and 1.5 nominal epochs. There are 233,455,616 trainable parameters. Only the final assistant body's reasoning, answer or tool call, and EOS contribute to loss. The maximum sequence length is 6,144; there is no cross-example packing. Overlong inherited examples are dropped whole rather than cutting off target answers.

The run completed 651 optimizer steps. Checkpoint 600 was selected by validation loss: **0.308386**, compared with **0.797465** before training. The adapter is merged at scale 1.0. The split contains 4,476,496 input tokens and 968,945 supervised tokens per nominal epoch. These figures describe the data and optimization, not downstream benchmark accuracy.

The release retains the native low, medium, and xhigh effort instructions. Its default effort is medium. A template compatibility fix normalizes historical inline reasoning and the `reasoning` alias once, preventing duplicated reasoning wrappers. Canonical rendering was checked across 48 native cases, plus historical compatibility and idempotence checks.

The selected generation settings are temperature 0.6, top-p 0.95, top-k 20, and repetition penalty 1.05. Reserved development coding tasks were used to compare native and compact-prompt variants and temperature 1.0. The compact instruction reduced medium-effort coding accuracy, so the release keeps the native effort prompts. Release results and sampling settings are reported separately from those development probes.

## Native MTP head

The main checkpoint contains all 15 native MTP tensors, including **424,699,392** head parameters. Head training freezes the entire verifier and uses its greedy next-token decisions as targets on supervised spans. The MTP input uses the backbone hidden state after final RMSNorm, matching the tested vLLM and llama.cpp implementations.

Training uses learning rate 5e-5, accumulation 4, and maximum length 2,048. The run processed 2,536 usable examples, **1,862,103 input tokens**, and 801,095 supervised tokens over 634 optimizer steps. Step 500 was selected for the best held-out verifier agreement among checkpoints whose validation loss did not exceed the starting head's loss.

On 48 held-out task-group rows containing 14,853 supervised positions, draft/verifier top-1 agreement increased from **90.4396% to 90.8975%**, and distillation loss decreased from **0.350240 to 0.296483**. These are held-out distillation measurements, not speculative acceptance rates or speedup claims. Runtime acceptance and speed are measured separately.

All 12 original backbone shard hashes were unchanged when the head was integrated. The safetensors index adds the 15 MTP tensors, and both the wrapper and text configuration identify one MTP layer. The release generation-default update also preserves all 13 final weight shard hashes.

Every text GGUF includes this head. Its matrix tensors retain Q8_0 precision even in lower-bit main-model quantizations. Conversion provenance and checksums identify the exact files.

## Compute and evaluation independence

All model generation, training, merging, quantization, and runtime testing occurred on Hugging Face Jobs. The user's local GPU was not used. Local operations were limited to orchestration, source editing, metadata checks, and analysis of saved outputs.

New training examples use training splits and separately authored tasks. HumanEval and MBPP development indices 0–31 are excluded from the reported release coding subsets. Exact issue IDs and normalized problem text from the 12 SWE-bench cases were checked against inherited replay with no matches. This does not exclude paraphrased overlap or foundation-pretraining exposure.

See `training_receipt.json`, `mtp_training.json`, `training_provenance.json`, and `evaluation_protocol.md` for machine-readable details and measurement limits.