# Changelog

Revision numbers identify published model states independently of release names.

## r006 — Pollock 1.4

- Returned from the r005 5B corpus to V2 of `SlayerLab/minimal-en-corpus-2.5b`, including its corrected cleaning and record boundaries.
- Restored the tokenizer used by r003-r004 and trained the unchanged 12/14/896 architecture for approximately 4.02 data passes at the same 491,520-token effective batch and 4e-4→4e-5 learning-rate range.
- Replaced the legacy inclusive training loop with the exact configured 22,003-update budget while preserving the training, data, and evaluation seeds.
- Updated training evaluation, the seven-task English zero-shot suite, Transformers weights, and fixed inference samples.

Full record: [`training-history/r006.md`](./training-history/r006.md)

## r005 — Pollock 1.3

- Returned to the 12/14/896 architecture used by r003 while keeping the model below 128M native trainable parameters.
- Replaced the 2.5B corpus with `SlayerLab/minimal-en-corpus-5b` and trained for two dataset passes at the same 491,520-token effective batch and 4e-4→4e-5 learning-rate range.
- Rebuilt the 12,288-entry byte-level BPE tokenizer for the new corpus while retaining the three special-token IDs.
- Updated training evaluation, the seven-task English zero-shot suite, and fixed inference samples.

Full record: [`training-history/r005.md`](./training-history/r005.md)

## r004 — Pollock 1.2

- Changed the architecture from 12/14/896 to 14/13/832 while keeping nearly the same parameter count.
- Kept the r003 tokenizer, dataset, context, effective batch, token budget, and 4e-4→4e-5 learning-rate range unchanged.
- Improved LAMBADA and SciQ over r003; the other five reported English primary metrics declined.
- Added a documentation-only, reproducible fixed-sampling suite for behavioral comparison of the immutable r001-r004 weights; model files are unchanged.

Full record: [`training-history/r004.md`](./training-history/r004.md)

## r003 — Pollock 1.1

- Kept the r002 architecture, tokenizer, dataset, context, effective batch, and four-pass token budget unchanged.
- Increased the learning-rate range from 3e-4→3e-5 to 4e-4→4e-5 and trained on 2× NVIDIA L40S.
- Improved all seven reported English zero-shot primary metrics over r002.

Full record: [`training-history/r003.md`](./training-history/r003.md)

## r002 — Pollock 1.0

- Expanded the architecture from 12/12/768 to 12/14/896 while remaining below 128M total parameters.
- Reduced context from 2,048 to 1,024 tokens and increased the effective batch from 262,144 to 491,520 tokens.
- Changed the learning-rate range from 6e-4→6e-5 to 3e-4→3e-5 and warmup from 500 to 440 iterations.
- Trained with BF16 on 2× RTX 4090 instead of 1× RTX 5090.
- Introduced sampled training validation and a seven-task English zero-shot benchmark suite.

Full record: [`training-history/r002.md`](./training-history/r002.md)

## r001 — experimental predecessor

- Initial experimental release with a 12/12/768 architecture, 2,048-token context, and 95.96M nanoGPT-reported parameters.
- Trained for four dataset passes and selected by deterministic full-validation reevaluation of retained checkpoints.
- No downstream benchmark suite was run.

Full record and generation samples: [`training-history/r001.md`](./training-history/r001.md)