# Changelog

Revision numbers identify published model states independently of release names.

## r004 — Pollock 1.2

- Changed the architecture from 12/14/896 to 14/13/832 while keeping nearly the same parameter count.
- Kept the r003 tokenizer, dataset, context, effective batch, token budget, and 4e-4→4e-5 learning-rate range unchanged.
- Improved LAMBADA and SciQ over r003; the other five reported English primary metrics declined.

Full record: [`training-history/r004.md`](./training-history/r004.md)

## r003 — Pollock 1.1

- Kept the r002 architecture, tokenizer, dataset, context, effective batch, and four-pass token budget unchanged.
- Increased the learning-rate range from 3e-4→3e-5 to 4e-4→4e-5 and trained on 2× NVIDIA L40S.
- Improved all seven reported English zero-shot primary metrics over r002.

Full record: [`training-history/r003.md`](./training-history/r003.md)

## r002 — Pollock 1.0

- Expanded the architecture from 12/12/768 to 12/14/896 while remaining below 128M total parameters.
- Reduced context from 2,048 to 1,024 tokens and increased the effective batch from 262,144 to 491,520 tokens.
- Changed the learning-rate range from 6e-4→6e-5 to 3e-4→3e-5 and warmup from 500 to 440 iterations.
- Trained with BF16 on 2× RTX 4090 instead of 1× RTX 5090.
- Introduced sampled training validation and a seven-task English zero-shot benchmark suite.

Full record: [`training-history/r002.md`](./training-history/r002.md)

## r001 — experimental predecessor

- Initial experimental release with a 12/12/768 architecture, 2,048-token context, and 95.96M nanoGPT-reported parameters.
- Trained for four dataset passes and selected by deterministic full-validation reevaluation of retained checkpoints.
- No downstream benchmark suite was run.

Full record and generation samples: [`training-history/r001.md`](./training-history/r001.md)