Instructions to use SlayerLab/pollock-mini-lm-125m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SlayerLab/pollock-mini-lm-125m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SlayerLab/pollock-mini-lm-125m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SlayerLab/pollock-mini-lm-125m")
model = AutoModelForCausalLM.from_pretrained("SlayerLab/pollock-mini-lm-125m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SlayerLab/pollock-mini-lm-125m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SlayerLab/pollock-mini-lm-125m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SlayerLab/pollock-mini-lm-125m

SGLang

How to use SlayerLab/pollock-mini-lm-125m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SlayerLab/pollock-mini-lm-125m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SlayerLab/pollock-mini-lm-125m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SlayerLab/pollock-mini-lm-125m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SlayerLab/pollock-mini-lm-125m with Docker Model Runner:
```
docker model run hf.co/SlayerLab/pollock-mini-lm-125m
```

dawidmajewski commited on 11 days ago

Commit

698984b

verified ·

1 Parent(s): 5e571cd

Release Pollock 1.1 (r003)

Browse files

Publish the verified r003 Transformers artifact, tokenizer, English benchmark results, model card, changelog, training history, and release manifest.

Files changed (7) hide show

CHANGELOG.md +8 -0
README.md +42 -42
benchmarks/english.json +0 -0
config.json +1 -1
model.safetensors +1 -1
release_manifest.json +24 -33
training-history/r003.md +133 -0

CHANGELOG.md CHANGED Viewed

@@ -2,6 +2,14 @@
 Revision numbers identify published model states independently of release names.
 ## r002 — Pollock 1.0
 - Expanded the architecture from 12/12/768 to 12/14/896 while remaining below 128M total parameters.

 Revision numbers identify published model states independently of release names.
+## r003 — Pollock 1.1
+- Kept the r002 architecture, tokenizer, dataset, context, effective batch, and four-pass token budget unchanged.
+- Increased the learning-rate range from 3e-4→3e-5 to 4e-4→4e-5 and trained on 2× NVIDIA L40S.
+- Improved all seven reported English zero-shot primary metrics over r002.
+Full record: [`training-history/r003.md`](./training-history/r003.md)
 ## r002 — Pollock 1.0
 - Expanded the architecture from 12/12/768 to 12/14/896 while remaining below 128M total parameters.

README.md CHANGED Viewed

@@ -8,35 +8,35 @@ license_link: https://huggingface.co/SlayerLab/pollock-mini-lm-125m/blob/main/LI
 datasets: [SlayerLab/minimal-en-corpus-2.5b]
 tags: [causal-lm, gpt2, nanogpt, bpe, educational, base-model]
 model-index:
-  - name: Pollock 1.0
     results:
       - task: {type: text-generation, name: Language modeling}
         dataset: {type: SlayerLab/minimal-en-corpus-2.5b, name: Minimal EN validation (sampled), split: validation}
-        metrics: [{type: loss, value: 2.577547, name: Final sampled validation loss}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: blimp, name: BLiMP, split: train}
-        metrics: [{type: acc, value: 0.7669701493}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: EleutherAI/lambada_openai, name: LAMBADA OpenAI, split: test}
-        metrics: [{type: acc, value: 0.2780904328}, {type: perplexity, value: 53.66752251}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: hellaswag, name: HellaSwag, split: validation}
-        metrics: [{type: acc_norm, value: 0.2987452699}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: piqa, name: PIQA, split: validation}
-        metrics: [{type: acc_norm, value: 0.6033732318}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: sciq, name: SciQ, split: test}
-        metrics: [{type: acc_norm, value: 0.658}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Easy, name: ARC-Easy, split: test}
-        metrics: [{type: acc_norm, value: 0.422979798}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Challenge, name: ARC-Challenge, split: test}
-        metrics: [{type: acc_norm, value: 0.2414675768}]
 ---
-# Pollock 1.0 — r002
 ![Pollock avatar](./assets/pollock-mini-lm-avatar-320.png)
@@ -46,15 +46,15 @@ model-index:
 ### Opis
-Pollock 1.0 (`r002`) to niewielki, anglojęzyczny model bazowy typu decoder-only, wytrenowany od zera jako czytelny eksperyment edukacyjny. Implementacja bazuje na [nanoGPT](https://github.com/karpathy/nanoGPT) i własnym tokenizerze byte-level BPE. Jest to model do uzupełniania tekstu, nie asystent konwersacyjny.
-Nazwa luźno nawiązuje do gestu malarskiego Jacksona Pollocka: nanoGPT jest płótnem, na którym dane, konfiguracja i decyzje treningowe tworzą różne wzorce zachowania. Pełne dane techniczne tej wersji znajdują się w [`training-history/r002.md`](./training-history/r002.md), a różnice między wydaniami w [`CHANGELOG.md`](./CHANGELOG.md).
 ### Architektura i tokenizer
 | Właściwość | Wartość |
 |---|---:|
-| Rewizja / wydanie | r002 / Pollock 1.0 |
 | Typ | decoder-only Transformer w stylu GPT-2 |
 | Warstwy / głowy / embedding | 12 / 14 / 896 |
 | Maksymalny kontekst | 1024 tokeny |
@@ -79,29 +79,29 @@ Model wytrenowano na [`SlayerLab/minimal-en-corpus-2.5b`](https://huggingface.co
 | Akumulacja globalna / na GPU | 40 / 20 micro-stepów |
 | Effective batch | 491 520 tokenów |
 | Optymalizator | fused AdamW, betas 0.9/0.95 |
-| Learning rate | 3e-4 → 3e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precyzja | BF16 |
-| Sprzęt | 2× NVIDIA GeForce RTX 4090 24 GB |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 ### Ewaluacja
-Loss treningowy szacowano na 100 losowo próbkowanych batchach na split. Finalny checkpoint uzyskał validation loss **2.577547**; najlepszy próbkowany wynik to **2.5600** w kroku 20 000. Nie należy porównywać tych wartości bezpośrednio z pełną, deterministyczną walidacją r001.
 Benchmarki wykonano zero-shot na pełnych splitach przy użyciu `lm-evaluation-harness` 0.4.12, batch size 8 i BF16.
 | Benchmark | Główna metryka | Wynik | Próbki |
 |---|---|---:|---:|
-| BLiMP | accuracy | 0.766970 | 67 000 |
-| LAMBADA OpenAI | accuracy | 0.278090 | 5 153 |
-| HellaSwag | normalized accuracy | 0.298745 | 10 042 |
-| PIQA | normalized accuracy | 0.603373 | 1 838 |
-| SciQ | normalized accuracy | 0.658000 | 1 000 |
-| ARC-Easy | normalized accuracy | 0.422980 | 2 376 |
-| ARC-Challenge | normalized accuracy | 0.241468 | 1 172 |
-LAMBADA osiągnęła perplexity 53.667523. Pełne metryki i protokół zapisano w historii r002.
 ### Użycie z Transformers
@@ -119,7 +119,7 @@ output = model.generate(
 print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
-Model używa standardowego `GPT2LMHeadModel`; `trust_remote_code=True` nie jest potrzebne. Dla r002 nie opublikowano jeszcze próbek generacji. Historyczne próbki są przypisane do poprzednich wag w historii r001.
 ### Ograniczenia i odpowiedzialne użycie
@@ -134,15 +134,15 @@ Model używa standardowego `GPT2LMHeadModel`; `trust_remote_code=True` nie jest
 ### Overview
-Pollock 1.0 (`r002`) is a small English decoder-only base language model trained from scratch as a readable educational experiment. It is based on [nanoGPT](https://github.com/karpathy/nanoGPT) and a custom byte-level BPE tokenizer. It is a completion model, not a conversational assistant.
-The name loosely refers to Jackson Pollock's painterly gesture: nanoGPT is the canvas on which data, configuration, and training decisions create different behavioral patterns. See [`training-history/r002.md`](./training-history/r002.md) for the complete technical record and [`CHANGELOG.md`](./CHANGELOG.md) for release-to-release changes.
 ### Architecture and tokenizer
 | Property | Value |
 |---|---:|
-| Revision / release | r002 / Pollock 1.0 |
 | Type | GPT-2-style decoder-only Transformer |
 | Layers / heads / width | 12 / 14 / 896 |
 | Maximum context | 1,024 tokens |
@@ -167,33 +167,33 @@ The model was trained on [`SlayerLab/minimal-en-corpus-2.5b`](https://huggingfac
 | Global / per-GPU accumulation | 40 / 20 micro-steps |
 | Effective batch | 491,520 tokens |
 | Optimizer | fused AdamW, betas 0.9/0.95 |
-| Learning rate | 3e-4 → 3e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precision | BF16 |
-| Hardware | 2× NVIDIA GeForce RTX 4090 24 GB |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 ### Evaluation
-Training-time loss was estimated from 100 randomly sampled batches per split. The final checkpoint achieved validation loss **2.577547**; the best sampled result was **2.5600** at step 20,000. These values are not directly comparable with r001's full deterministic validation.
-Benchmarks used complete splits with `lm-evaluation-harness` 0.4.12, zero-shot evaluation, batch size 8, and BF16.
 | Benchmark | Primary metric | Score | Samples |
 |---|---|---:|---:|
-| BLiMP | accuracy | 0.766970 | 67,000 |
-| LAMBADA OpenAI | accuracy | 0.278090 | 5,153 |
-| HellaSwag | normalized accuracy | 0.298745 | 10,042 |
-| PIQA | normalized accuracy | 0.603373 | 1,838 |
-| SciQ | normalized accuracy | 0.658000 | 1,000 |
-| ARC-Easy | normalized accuracy | 0.422980 | 2,376 |
-| ARC-Challenge | normalized accuracy | 0.241468 | 1,172 |
-LAMBADA perplexity was 53.667523. Full metrics and protocol details are recorded in the r002 history.
 ### Usage
-Use the Transformers example in the Polish section. The artifact uses standard `GPT2LMHeadModel`; `trust_remote_code=True` is unnecessary. No r002 generation samples have been published yet. Historical samples are explicitly attached to the predecessor weights in the r001 history.
 ### Limitations and responsible use
@@ -208,7 +208,7 @@ Use the Transformers example in the Polish section. The artifact uses standard `
 Kod nanoGPT jest na licencji MIT. Korpus łączy źródła o różnych warunkach i nie ma jednej wspólnej licencji, dlatego model oznaczono jako `license: other`. Szczegóły: [`LICENSE.md`](./LICENSE.md) i karta datasetu.
-nanoGPT code is MIT-licensed. The corpus combines sources governed by different terms and has no single common license, so the model is marked `license: other`. See [`LICENSE.md`](./LICENSE.md) and the dataset card.
 ## Pochodzenie / Attribution

 datasets: [SlayerLab/minimal-en-corpus-2.5b]
 tags: [causal-lm, gpt2, nanogpt, bpe, educational, base-model]
 model-index:
+  - name: Pollock 1.1
     results:
       - task: {type: text-generation, name: Language modeling}
         dataset: {type: SlayerLab/minimal-en-corpus-2.5b, name: Minimal EN validation (sampled), split: validation}
+        metrics: [{type: loss, value: 2.5528755188, name: Final sampled validation loss}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: blimp, name: BLiMP, split: train}
+        metrics: [{type: acc, value: 0.7773880597}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: EleutherAI/lambada_openai, name: LAMBADA OpenAI, split: test}
+        metrics: [{type: acc, value: 0.2899281972}, {type: perplexity, value: 47.70114023}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: hellaswag, name: HellaSwag, split: validation}
+        metrics: [{type: acc_norm, value: 0.3034256124}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: piqa, name: PIQA, split: validation}
+        metrics: [{type: acc_norm, value: 0.6109902067}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: sciq, name: SciQ, split: test}
+        metrics: [{type: acc_norm, value: 0.659}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Easy, name: ARC-Easy, split: test}
+        metrics: [{type: acc_norm, value: 0.428030303}]
       - task: {type: text-generation, name: Zero-shot evaluation}
         dataset: {type: allenai/ai2_arc, config: ARC-Challenge, name: ARC-Challenge, split: test}
+        metrics: [{type: acc_norm, value: 0.2534129693}]
 ---
+# Pollock 1.1 — r003
 ![Pollock avatar](./assets/pollock-mini-lm-avatar-320.png)
 ### Opis
+Pollock to niewielki, anglojęzyczny model bazowy typu decoder-only, wytrenowany od zera jako czytelny eksperyment edukacyjny. Implementacja bazuje na [nanoGPT](https://github.com/karpathy/nanoGPT) i własnym tokenizerze byte-level BPE. Jest to model do uzupełniania tekstu, nie asystent konwersacyjny.
+Nazwa luźno nawiązuje do gestu malarskiego Jacksona Pollocka: nanoGPT jest płótnem, na którym dane, konfiguracja i decyzje treningowe tworzą różne wzorce zachowania. Pełne dane techniczne tej wersji znajdują się w [`training-history/r003.md`](./training-history/r003.md), a różnice między wydaniami w [`CHANGELOG.md`](./CHANGELOG.md).
 ### Architektura i tokenizer
 | Właściwość | Wartość |
 |---|---:|
+| Rewizja / wydanie | r003 / Pollock 1.1 |
 | Typ | decoder-only Transformer w stylu GPT-2 |
 | Warstwy / głowy / embedding | 12 / 14 / 896 |
 | Maksymalny kontekst | 1024 tokeny |
 | Akumulacja globalna / na GPU | 40 / 20 micro-stepów |
 | Effective batch | 491 520 tokenów |
 | Optymalizator | fused AdamW, betas 0.9/0.95 |
+| Learning rate | 4e-4 → 4e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precyzja | BF16 |
+| Sprzęt | 2× NVIDIA L40S |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 ### Ewaluacja
+Loss treningowy szacowano na stałych podzbiorach po 1 228 800 tokenów na split. Finalny checkpoint uzyskał validation loss **2.552876**; najlepszy wynik to **2.5362** w kroku 20 000. Nie należy porównywać tych wartości bezpośrednio z losowo próbkowaną walidacją r002.
 Benchmarki wykonano zero-shot na pełnych splitach przy użyciu `lm-evaluation-harness` 0.4.12, batch size 8 i BF16.
 | Benchmark | Główna metryka | Wynik | Próbki |
 |---|---|---:|---:|
+| BLiMP | accuracy | 0.777388 | 67 000 |
+| LAMBADA OpenAI | accuracy | 0.289928 | 5 153 |
+| HellaSwag | normalized accuracy | 0.303426 | 10 042 |
+| PIQA | normalized accuracy | 0.610990 | 1 838 |
+| SciQ | normalized accuracy | 0.659000 | 1 000 |
+| ARC-Easy | normalized accuracy | 0.428030 | 2 376 |
+| ARC-Challenge | normalized accuracy | 0.253413 | 1 172 |
+LAMBADA osiągnęła perplexity 47.701140. Pełne metryki i protokół zapisano w historii r003.
 ### Użycie z Transformers
 print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
+Model używa standardowego `GPT2LMHeadModel`; `trust_remote_code=True` nie jest potrzebne. Dla r003 nie opublikowano jeszcze próbek generacji. Historyczne próbki są przypisane do poprzednich wag w historii r001.
 ### Ograniczenia i odpowiedzialne użycie
 ### Overview
+Pollock is a small English decoder-only base language model trained from scratch as a readable educational experiment. It is based on [nanoGPT](https://github.com/karpathy/nanoGPT) and a custom byte-level BPE tokenizer. It is a completion model, not a conversational assistant.
+The name loosely refers to Jackson Pollock's painterly gesture: nanoGPT is the canvas on which data, configuration, and training decisions create different behavioral patterns. See [`training-history/r003.md`](./training-history/r003.md) for the complete technical record and [`CHANGELOG.md`](./CHANGELOG.md) for release-to-release changes.
 ### Architecture and tokenizer
 | Property | Value |
 |---|---:|
+| Revision / release | r003 / Pollock 1.1 |
 | Type | GPT-2-style decoder-only Transformer |
 | Layers / heads / width | 12 / 14 / 896 |
 | Maximum context | 1,024 tokens |
 | Global / per-GPU accumulation | 40 / 20 micro-steps |
 | Effective batch | 491,520 tokens |
 | Optimizer | fused AdamW, betas 0.9/0.95 |
+| Learning rate | 4e-4 → 4e-5, cosine decay |
 | Warmup / weight decay / grad clip | 440 / 0.1 / 1.0 |
 | Precision | BF16 |
+| Hardware | 2× NVIDIA L40S |
 | Framework | PyTorch 2.8.0+cu128, nanoGPT commit `3adf61e` |
 ### Evaluation
+Training-time loss was estimated on fixed subsets of 1,228,800 tokens per split. The final checkpoint achieved validation loss **2.552876**; the best result was **2.5362** at step 20,000. These values are not directly comparable with r002's randomly sampled training validation.
+Benchmarks used complete splits with `lm-evaluation-harness` 0.4.12, zero few-shot examples, batch size 8, and BF16.
 | Benchmark | Primary metric | Score | Samples |
 |---|---|---:|---:|
+| BLiMP | accuracy | 0.777388 | 67,000 |
+| LAMBADA OpenAI | accuracy | 0.289928 | 5,153 |
+| HellaSwag | normalized accuracy | 0.303426 | 10,042 |
+| PIQA | normalized accuracy | 0.610990 | 1,838 |
+| SciQ | normalized accuracy | 0.659000 | 1,000 |
+| ARC-Easy | normalized accuracy | 0.428030 | 2,376 |
+| ARC-Challenge | normalized accuracy | 0.253413 | 1,172 |
+LAMBADA perplexity was 47.701140. Full metrics and protocol details are recorded in the r003 history.
 ### Usage
+Use the Transformers example in the Polish section. The artifact uses standard `GPT2LMHeadModel`; `trust_remote_code=True` is unnecessary. No r003 generation samples have been published yet. Historical samples are explicitly attached to the predecessor weights in the r001 history.
 ### Limitations and responsible use
 Kod nanoGPT jest na licencji MIT. Korpus łączy źródła o różnych warunkach i nie ma jednej wspólnej licencji, dlatego model oznaczono jako `license: other`. Szczegóły: [`LICENSE.md`](./LICENSE.md) i karta datasetu.
+nanoGPT code is MIT-licensed. The corpus combines sources governed by different terms and has no single common license, so the model is marked as `license: other`. See [`LICENSE.md`](./LICENSE.md) and the dataset card.
 ## Pochodzenie / Attribution

benchmarks/english.json CHANGED Viewed

The diff for this file is too large to render. See raw diff

config.json CHANGED Viewed

@@ -12,7 +12,7 @@
   "initializer_range": 0.02,
   "layer_norm_epsilon": 1e-05,
   "model_type": "gpt2",
-  "model_version": "1.0",
   "n_ctx": 1024,
   "n_embd": 896,
   "n_head": 14,

   "initializer_range": 0.02,
   "layer_norm_epsilon": 1e-05,
   "model_type": "gpt2",
+  "model_version": "1.1",
   "n_ctx": 1024,
   "n_embd": 896,
   "n_head": 14,

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2aada5b26abe9ce3b70af393d77be4ed1a799a7205b9c0dd5cfaff8bcaabce68
 size 510713512

 version https://git-lfs.github.com/spec/v1
+oid sha256:3a9964a0ba5f6d20ef6061d2a7e4442c1b001209ebe80f7b7148a25573decaa9
 size 510713512

release_manifest.json CHANGED Viewed

@@ -1,12 +1,12 @@
 {
   "schema_version": 2,
-  "revision": 2,
-  "revision_id": "r002",
-  "release": "Pollock 1.0",
   "model_id": "SlayerLab/pollock-mini-lm-125m",
   "source_checkpoint": {
-    "path_in_training_workspace": "runs/minimal-en-125m-4ep/ckpt.pt",
-    "sha256": "026f54a390036b35792aa8fb131c8b0d394efb6d933753fb0e1257e55a67374b",
     "iteration": 22004,
     "tokens_seen": 10815406080,
     "native_nanogpt_parameters": 126637952,
@@ -30,8 +30,8 @@
     "ddp_world_size": 2,
     "effective_batch_tokens": 491520,
     "optimizer": "fused AdamW",
-    "learning_rate": 0.0003,
-    "min_learning_rate": 0.00003,
     "schedule": "cosine",
     "warmup_iters": 440,
     "lr_decay_iters": 22003,
@@ -43,26 +43,21 @@
     "compile": true,
     "backend": "nccl",
     "seed": 1337,
-    "hardware": "2x NVIDIA GeForce RTX 4090 24 GB",
-    "nanogpt_commit": "3adf61e"
   },
   "evaluation": {
-    "final_sampled_validation_loss": 2.577547,
-    "best_sampled_validation_loss": 2.56,
-    "best_sampled_validation_step": 20000,
-    "training_eval_batches": 100,
     "benchmark_harness": "lm-evaluation-harness 0.4.12",
     "benchmark_num_fewshot": 0,
     "benchmark_batch_size": 8,
-    "benchmarks": {
-      "blimp": {"acc": 0.7669701492537313, "samples": 67000},
-      "lambada_openai": {"acc": 0.2780904327576169, "perplexity": 53.66752251060021, "samples": 5153},
-      "hellaswag": {"acc_norm": 0.29874526986656047, "acc": 0.2818163712407887, "samples": 10042},
-      "piqa": {"acc_norm": 0.6033732317736671, "acc": 0.6137105549510338, "samples": 1838},
-      "sciq": {"acc_norm": 0.658, "acc": 0.737, "samples": 1000},
-      "arc_easy": {"acc_norm": 0.42297979797979796, "acc": 0.4659090909090909, "samples": 2376},
-      "arc_challenge": {"acc_norm": 0.24146757679180889, "acc": 0.20819112627986347, "samples": 1172}
-    }
   },
   "conversion": {
     "target_class": "GPT2LMHeadModel",
@@ -73,19 +68,15 @@
     "max_absolute_logit_error": 0.0
   },
   "artifacts": {
-    "README.md": {"sha256": "789645020a3864ad81e8a5271118aeed322c9a56039209e715689866d781342f"},
-    "CHANGELOG.md": {"sha256": "379bf0c49f8a8c4a2d40e3ae1775876c37de2e6dac8125385be77742c96529d0"},
-    "training-history/r001.md": {"sha256": "5e3cb7664fa09902369bb650d1f94cc63b1f0bd1c8a7ffea5c1e6c16c8c4c5df"},
-    "training-history/r002.md": {"sha256": "bf4364b27d966fe32676d75c269dfd1e53a4939246a0888c0cce7cd63641952d"},
-    "benchmarks/english.json": {"sha256": "eff82f7d45c82d715c536ac728787ea1eee42515501d4a0544934badab69ce2d"},
-    "LICENSE.md": {"sha256": "46cbe928ed0aa24875f02f774313b27ec9d8abdf41f0c9ef67e0adb4e0614de4"},
-    "config.json": {"sha256": "0ad9e47efb8d2ddf4666f016a286604365faa4ff6ece9654718c3e4daa43c41e"},
     "generation_config.json": {"sha256": "435beb27be51f0ed054f4a011e5109d125cdadc118b8799b18b155cc798d94d2"},
-    "model.safetensors": {"sha256": "2aada5b26abe9ce3b70af393d77be4ed1a799a7205b9c0dd5cfaff8bcaabce68"},
     "special_tokens_map.json": {"sha256": "8b2257a17ea997bb038f43b133aefec82344ad2b8abc2b8a02a6c0a994ed624e"},
     "tokenizer.json": {"sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51"},
-    "tokenizer_config.json": {"sha256": "4cdabe37dbdc1adfcc017ee9a1f86ab89bdf184827d2d9f05181cae0f8af19bf"},
-    "logs/training.log": {"sha256": "ff856088bdf504759600553a4e9ad5a23f965d54646838adcf0599bb1621d6e1"},
-    "logs/benchmark-english.log": {"sha256": "f81ad6c915ae49a0d49f61c15ea86a265c31364e5418150c35aa416d4809e22a"}
   }
 }

 {
   "schema_version": 2,
+  "revision": 3,
+  "revision_id": "r003",
+  "release": "Pollock 1.1",
   "model_id": "SlayerLab/pollock-mini-lm-125m",
   "source_checkpoint": {
+    "path_in_training_workspace": "runs/r003-lr4e-4-v4/checkpoints/ckpt.pt",
+    "sha256": "912483b44bb177c7a62fe19a53fc6db7b333cf881c57de15daf9dd7209d90a6b",
     "iteration": 22004,
     "tokens_seen": 10815406080,
     "native_nanogpt_parameters": 126637952,
     "ddp_world_size": 2,
     "effective_batch_tokens": 491520,
     "optimizer": "fused AdamW",
+    "learning_rate": 0.0004,
+    "min_learning_rate": 0.00004,
     "schedule": "cosine",
     "warmup_iters": 440,
     "lr_decay_iters": 22003,
     "compile": true,
     "backend": "nccl",
     "seed": 1337,
+    "evaluation_seed": 4242,
+    "hardware": "2x NVIDIA L40S",
+    "nanogpt_commit": "3adf61e154c3fe3fca428ad6bc3818b27a3b8291"
   },
   "evaluation": {
+    "protocol": "fixed sampled subset",
+    "subset_tokens_per_split": 1228800,
+    "final_validation_loss": 2.552875518798828,
+    "best_validation_loss": 2.5362,
+    "best_validation_step": 20000,
     "benchmark_harness": "lm-evaluation-harness 0.4.12",
     "benchmark_num_fewshot": 0,
     "benchmark_batch_size": 8,
+    "truncated_benchmark_requests": 80,
+    "results_file": "benchmarks/english.json"
   },
   "conversion": {
     "target_class": "GPT2LMHeadModel",
     "max_absolute_logit_error": 0.0
   },
   "artifacts": {
+    "README.md": {"sha256": "5ab395ed94618c311257fed4c16c40ed558919e6b0b0f12177eca4250dc41f5a"},
+    "CHANGELOG.md": {"sha256": "00a60accbe6dfe0734326124c789f842141f61ecc26d7cd2f8291b148bfd125f"},
+    "training-history/r003.md": {"sha256": "4ae07419880227300a5c6c2ae259d7f609d928f33ab9c0c91cbce44dc6b56428"},
+    "benchmarks/english.json": {"sha256": "7dd602e81a79bbdad4393b9031d0f584148076310d85cbda71fb6867b1ad63d8"},
+    "config.json": {"sha256": "89816fc9a57e4915ef884864dfb7b0502dbc16e3da749e781ce41b2a2c7ce26d"},
     "generation_config.json": {"sha256": "435beb27be51f0ed054f4a011e5109d125cdadc118b8799b18b155cc798d94d2"},
+    "model.safetensors": {"sha256": "3a9964a0ba5f6d20ef6061d2a7e4442c1b001209ebe80f7b7148a25573decaa9"},
     "special_tokens_map.json": {"sha256": "8b2257a17ea997bb038f43b133aefec82344ad2b8abc2b8a02a6c0a994ed624e"},
     "tokenizer.json": {"sha256": "6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51"},
+    "tokenizer_config.json": {"sha256": "4cdabe37dbdc1adfcc017ee9a1f86ab89bdf184827d2d9f05181cae0f8af19bf"}
   }
 }

training-history/r003.md ADDED Viewed

	@@ -0,0 +1,133 @@

+# r003 - Pollock 1.1 training record
+## Identity and provenance
+| Field | Value |
+|---|---|
+| Revision / release | `r003` / Pollock 1.1 |
+| Model ID | `SlayerLab/pollock-mini-lm-125m` |
+| Source run | `runs/r003-lr4e-4-v4` |
+| Source checkpoint | `checkpoints/ckpt.pt` |
+| Checkpoint iteration / SHA-256 | 22,004 / `912483b44bb177c7a62fe19a53fc6db7b333cf881c57de15daf9dd7209d90a6b` |
+| Benchmark adapter SHA-256 | `0f53aa6b8454e856f8c4c489c7e6cbf207b8e124fe06bbb239315c017fe16fc0` |
+| nanoGPT commit | `3adf61e154c3fe3fca428ad6bc3818b27a3b8291` (dirty workspace recorded) |
+| W&B run | [`uro1w8l1`](https://wandb.ai/dawidmajewski-ezotic/minimal-en/runs/uro1w8l1) |
+| Training environment | [DMPod GPT](https://github.com/dawidmajewski/dmpod-gpt) on RunPod |
+| Run status | completed, exit code 0 |
+The native checkpoint reports `iter_num=22004`. The wrapper's pre-run estimate in `config.json` used 22,003 target updates and is one effective batch lower because the legacy nanoGPT loop is inclusive. This record follows the native checkpoint and r002 release convention: 22,004 updates and 10,815,406,080 token presentations.
+## Architecture and parameter accounting
+| Setting | Value |
+|---|---:|
+| Type | GPT-2-style decoder-only Transformer |
+| Layers / heads / width | 12 / 14 / 896 |
+| Head dimension / MLP width | 64 / 3,584 |
+| Context / vocabulary | 1,024 / 12,288 |
+| Activation / dropout / bias | GELU / 0.0 / false |
+| Position representation | learned absolute embeddings |
+| Input/output token weights | tied |
+| nanoGPT reported count | 126,637,952 |
+| Native unique trainable total | 127,555,456 |
+The architecture and parameter counts are unchanged from r002. The Transformers artifact has 127,674,624 unique serialized parameters, including 119,168 zero-valued compatibility bias parameters.
+## Tokenizer and dataset
+The custom tokenizer is byte-level BPE with GPT-2-style pretokenization, 12,288 vocabulary entries, and the special tokens `<|endoftext|>`, `<|im_start|>`, and `<|im_end|>`. Tokenizer SHA-256: `6cda4e5ec8293f3b821e02253f9c0e88e43ad4f7c6e1324763e1fb91749fef51`.
+Training used [Minimal EN Corpus 2.5B](https://huggingface.co/datasets/SlayerLab/minimal-en-corpus-2.5b).
+| Split | BPE tokens | Binary SHA-256 |
+|---|---:|---|
+| Train | 2,703,768,815 | `5396834c25664080c14793a4880f86b86ae94074a6036f8f54a58686b02d30dc` |
+| Validation | 5,267,742 | `bf07bd1fb1f956a92ee7788f2a8110e1837f5b880ce432f6cd2a22d05f8ac135` |
+Dataset and tokenizer identities match r002.
+## Complete training configuration
+| Setting | Value |
+|---|---:|
+| Initialization | scratch |
+| Final checkpoint iteration | 22,004 |
+| Sequence / micro-batch per GPU | 1,024 / 12 |
+| Global / per-GPU accumulation | 40 / 20 micro-steps |
+| DDP world size / effective batch | 2 / 491,520 tokens |
+| Token presentations / passes | 10,815,406,080 / approximately 4.0 |
+| Optimizer / betas | fused AdamW / 0.9, 0.95 |
+| Maximum / minimum LR | 4e-4 / 4e-5 |
+| Schedule / warmup / decay | cosine / 440 / 22,003 iterations |
+| Weight decay / grad clip | 0.1 / 1.0 |
+| Evaluation interval / batches | 1,000 / 100 per split |
+| Evaluation subset | fixed 1,228,800 tokens per split |
+| Training / data / evaluation seeds | 1,337 / 1,337 / 4,242 |
+| Precision / compilation / backend | BF16 / `torch.compile` / NCCL |
+| Hardware | 2x NVIDIA L40S |
+| PyTorch / CUDA | 2.8.0+cu128 / 12.8 |
+| Runtime | approximately 11 h 40 min |
+The controlled change from r002 was the maximum/minimum learning rate. Hardware and the evaluation sampling implementation also differed operationally.
+## Training-time validation
+Loss estimates used fixed offset files containing 1,200 sequence starts per split. They are internally comparable within r003 but not directly comparable with r002's randomly sampled batches or r004's differently seeded offsets.
+| Step | Train loss | Validation loss |
+|---:|---:|---:|
+| 0 | 9.5797 | 9.5732 |
+| 1,000 | 3.3896 | 3.4346 |
+| 2,000 | 3.0458 | 3.0779 |
+| 3,000 | 2.8829 | 2.9362 |
+| 4,000 | 2.8210 | 2.8694 |
+| 5,000 | 2.7688 | 2.8231 |
+| 6,000 | 2.6991 | 2.7486 |
+| 7,000 | 2.6873 | 2.7129 |
+| 8,000 | 2.6536 | 2.6828 |
+| 9,000 | 2.6451 | 2.6794 |
+| 10,000 | 2.5838 | 2.6674 |
+| 11,000 | 2.5810 | 2.6154 |
+| 12,000 | 2.5351 | 2.6060 |
+| 13,000 | 2.5606 | 2.5888 |
+| 14,000 | 2.5332 | 2.5935 |
+| 15,000 | 2.5207 | 2.5801 |
+| 16,000 | 2.4850 | 2.5726 |
+| 17,000 | 2.4773 | 2.5945 |
+| 18,000 | 2.4851 | 2.5548 |
+| 19,000 | 2.4698 | 2.5561 |
+| 20,000 | 2.4437 | **2.5362** |
+| 21,000 | 2.4602 | 2.5417 |
+| 22,000 | **2.4282** | 2.5529 |
+The final W&B value is 2.5528755188 with perplexity 12.8439838496. The released source checkpoint is the final checkpoint, not the step-20,000 sampled minimum.
+## English zero-shot benchmarks
+Tasks used complete splits with `lm-evaluation-harness` 0.4.12, zero few-shot examples, batch size 8, BF16, and 1,024 maximum context. The adapter recorded 80 requests truncated to the model context.
+| Benchmark | Primary | Additional | Samples | Delta vs r002 |
+|---|---:|---:|---:|---:|
+| BLiMP | acc 0.7773880597 | - | 67,000 | +0.010418 |
+| LAMBADA OpenAI | acc 0.2899281972 | perplexity 47.7011402302 | 5,153 | +0.011838 acc |
+| HellaSwag | acc_norm 0.3034256124 | acc 0.2850029875 | 10,042 | +0.004680 |
+| PIQA | acc_norm 0.6109902067 | acc 0.6147986942 | 1,838 | +0.007617 |
+| SciQ | acc_norm 0.6590000000 | acc 0.7660000000 | 1,000 | +0.001000 |
+| ARC-Easy | acc_norm 0.4280303030 | acc 0.4861111111 | 2,376 | +0.005051 |
+| ARC-Challenge | acc_norm 0.2534129693 | acc 0.2201365188 | 1,172 | +0.011945 |
+All seven primary scores improved over r002. LAMBADA perplexity decreased by 5.966382. Because the r002 artifact reported zero truncated requests, the cross-revision comparison is informative but not a byte-for-byte identical evaluation protocol.
+Structured English results: [`../benchmarks/english.json`](../benchmarks/english.json).
+## Release preparation status
+- Native training checkpoint verified against the backup archive.
+- Benchmark adapter hash verified against the benchmark result record.
+- Dataset, tokenizer, source snapshots, and fixed evaluation offsets are present.
+- Transformers 5.15.1 conversion is complete; the `[2, 64]` deterministic parity probe produced a maximum absolute logit error of 0.0, and artifact hashes are recorded in `release_manifest.json`.
+- No files have been uploaded to Hugging Face for this revision.
+## Licensing and limitations
+This is an English, unaligned base model, not an instruction-following assistant. It may hallucinate, generate harmful or biased content, and reproduce training patterns. It is unsuitable for high-risk or production decisions without independent safeguards. The dataset combines sources with different terms; see the dataset card and the model repository's `LICENSE.md`.