---
license: apache-2.0
base_model: kitft/nla-qwen2.5-7b-L20-av
tags: [nla, natural-language-autoencoders, qwen2.5, av]
---
# nla-qwen2.5-7b-L20-av-matryoshka-sonnet46-v3 — v3 warm-start (bullets prompt)

Actor (AV) of the [NLA](https://github.com/kitft/natural_language_autoencoders) Qwen2.5-7B
(layer-20) pair, **v3 warm-start**: SFT from the base
[`kitft/nla-qwen2.5-7b-L20-av`](https://huggingface.co/kitft/nla-qwen2.5-7b-L20-av)
on Claude Sonnet-4.6 explanations in the v3 format.

v3 changes vs v2 (see the repo's experiment README):
- The prompt actually says **bullet points** ("a list of bullet points, one per line");
  output text is plain one-item-per-line (no literal "- " markers). The `nla_meta.yaml`
  sidecar carries the real trained template (v1/v2 shipped a stale tagged one).
- AV: trained with NLA_NO_TRAIN_EOS=1 (never taught to stop; RL caps generation).
- Intended as the starting point for the v3 RL phase (KL 0.03, uniform token
  truncation ~U[1,120]).

Warm-start data: [v3 splits](https://huggingface.co/datasets/syvb/nla-qwen2.5-7b-L20-matryoshka-warmstart-sonnet46) (`v3/` folder).
Hyperparameters: global batch 256, lr 2e-5→2e-6 cosine, warmup 50, 1 epoch, injection_scale 150.