spandyie/amadablam-322m-instruct

Ama Dablam 322M DPO

DPO-tuned Ama Dablam, starting from the instruction-tuned checkpoint: preference-trained to prefer well-formed, on-script, appropriately concise responses over degenerate ones (run-on translations, script drift, near-duplicate low-signal pairs). Trained in two stages — a short DPO-format SFT warmup, then DPO proper (beta=0.1) against a frozen reference — on 14,152 preference pairs across Nepali, Maithili, and Bhojpuri, rendered 85/5/10 across Devanagari/IAST/phonetic script.

metric	value
preference accuracy (val, n=744)	91.4% (mean margin +4.85)
— by language	ne 94.9% · mai 91.0% · bho 87.3%
— by script	deva 92.8% · phon 86.3% · iast 77.1%
script fidelity (replies in the prompt's script)	95% (19/20 sampled)
response length	mean 32.3 tokens, 95% properly terminated

Forgetting guardrail (drift vs. the pre-DPO checkpoint on 6 pinned pretrain val shards + phonetic-ne) passed for every metric, but drift was uniformly positive (a small, one-directional erosion, not noise): worst cases IAST-Maithili +0.035 bpb, phonetic-Nepali +0.027 bpb; Devanagari ne/mai/bho drift stayed under +0.01 bpb.

Usage (chat template built in — identical to the instruct model)

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("spandyie/amadablam-322m-dpo")
model = AutoModelForCausalLM.from_pretrained("spandyie/amadablam-322m-dpo",
                                             trust_remote_code=True)
msgs = [{"role": "user", "content": "स्वस्थ रहन के गर्नुपर्छ?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=200)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Limitations (read before use)

• License: CC-BY-NC-4.0 (non-commercial) — inherited from the instruct checkpoint's Alpaca/Dolly-family lineage, and independently required by the Bhojpuri leg of this model's own DPO data (Bhojpuri routes through NLLB for translation, which is CC-BY-NC-4.0 and propagates to anything built with it).
• 322M params: same factual-reliability caveats as the base/instruct models — this stage tunes response quality/preference, not knowledge.
• Bhojpuri and IAST are the weakest legs (87.3% / 77.1% preference accuracy vs. 91-95% elsewhere) — this tracks thinner/lower-quality source data for those combinations earlier in the pipeline, not a DPO-specific defect.
• Forgetting drift, while within the pass tolerance, moved consistently in one direction across all pinned shards — treat this checkpoint as a first DPO iteration, not a final answer, if you plan to iterate further.
• This model is cacheless by design. generation_config.json sets use_cache: false. Overriding to use_cache=True silently produces incoherent, context-free output (only the last token is fed back each step) rather than raising an error.
• Right-pad with BOS id 16368. No system role. Multi-turn lightly trained.