ivykopal/nlp-2026-hanoi-sft-50

SmolLM2-135M · Tower of Hanoi (flat SFT, full fine-tuning)

A full fine-tune of HuggingFaceTB/SmolLM2-135M-Instruct that produces a full Tower of Hanoi solution in a single generation — one move per line, e.g. A->C. The FFT baseline from the NLP 2026 fine-tuning / RL comparison (examples/sft/fft.ipynb).

This is the flat format baseline. It is trained to imitate optimal trajectories, so it inherits the per-move error of its demonstrations and exhibits a hard length prior: trained on at most 31-move (5-disk) solutions, it emits exactly 31 legal moves for a 63-move (6-disk) puzzle and stops. That is why extrapolation solved-rate is 0%.

Training

	value
Base model	SmolLM2-135M-Instruct
Method	full fine-tuning (FFT)
Label noise	20% corrupted trajectories
Train disk counts	3–5
Saved with	transformers 4.57.6

The supervised targets carry 20% corrupted demonstrations: an optimal trajectory with one move swapped, replaced, deleted or inserted. The model can only imitate that distribution, so it inherits a per-move error rate that compounds over the move chain.

Evaluation (greedy, one generation per puzzle)

disks	heldout solved	extrapolation solved
3	0.79	—
4	0.36	—
5	0.67	—
6	—	0.00
overall	0.60	0.00

solved = the trajectory moved every disk to the target peg.

Heldout (3–5 disks, unseen peg permutations): 0.60 solved. Extrapolation (6 disks, strictly longer than training): 0.00 solved. This is the length-prior failure mode, not accumulated error — the model emits a well-formed, fully legal answer of the wrong length and stops.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ivykopal/nlp-2026-hanoi-sft")
tokenizer = AutoTokenizer.from_pretrained("ivykopal/nlp-2026-hanoi-sft")

system = (
    "You are an expert algorithmic problem solver. "
    "Solve the Tower of Hanoi puzzle optimally. "
    "Return ONLY one move per line in the format 'A->C'. "
    "Do not provide any explanation."
)
# Move 4 disks from peg A to peg C using peg B.
user = (
    "Solve the Tower of Hanoi puzzle.\n\n"
    "Move 4 disks from peg A to peg C using peg B.\n"
    "Return only the sequence of moves."
)

messages = [{"role": "system", "content": system}, {"role": "user", "content": user}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=576, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
# A->B
# A->C
# ...

Decode greedily (do_sample=False). max_new_tokens must fit the longest solution you evaluate — 6 disks needs 63 moves (~251 tokens).

Intended use

Research and education: a tiny, fully reproducible fine-tune for studying demonstration-noise propagation, length-extrapolation failure, and as the SFT baseline / cold start for GRPO/GSPO. Not a general assistant — it only produces Tower of Hanoi move sequences.

License

Apache-2.0 (inherited from the base model).