---
license: apache-2.0
base_model: kitft/nla-qwen2.5-7b-L20-ar
tags: [nla, natural-language-autoencoders, qwen2.5, ar]
---
# nla-qwen2.5-7b-L20-ar-matryoshka-sonnet46-v3 — v3 warm-start (bullets prompt)

Critic (AR) of the [NLA](https://github.com/kitft/natural_language_autoencoders) Qwen2.5-7B
(layer-20) pair, **v3 warm-start**: SFT from the base
[`kitft/nla-qwen2.5-7b-L20-ar`](https://huggingface.co/kitft/nla-qwen2.5-7b-L20-ar)
on Claude Sonnet-4.6 explanations in the v3 format.

v3 changes vs v2 (see the repo's experiment README):
- The prompt actually says **bullet points** ("a list of bullet points, one per line");
  output text is plain one-item-per-line (no literal "- " markers). The `nla_meta.yaml`
  sidecar carries the real trained template (v1/v2 shipped a stale tagged one).
- AR: critic inputs token-truncated to K ~ U[1,120] (2% left full) — pre-calibrated for the RL-time uniform token truncation.
- Intended as the starting point for the v3 RL phase (KL 0.03, uniform token
  truncation ~U[1,120]).

Warm-start data: [v3 splits](https://huggingface.co/datasets/syvb/nla-qwen2.5-7b-L20-matryoshka-warmstart-sonnet46) (`v3/` folder).
Hyperparameters: global batch 256, lr 2e-5→2e-6 cosine, warmup 50, 1 epoch.