# Byrne-100M-Ultra-MC — provenance

Byrne / SpikeWhale v2. Looped (`loop_count=3`), dense, 113.9M params, plus a
parallel Memory Cache branch (arXiv:2602.24281).

## Checkpoints

| release file | came from | stage |
|---|---|---|
| `checkpoints/base_62k.pt` | `checkpoints_100m_ultrax_mc/step_00062000.pt` | pretrain, Dolma-blend continuation, step 62k |
| `checkpoints/sft_7100.pt`  | `sft_if_step_00007100.pt` | SFT on HuggingFaceH4/ultrachat_200k, step 7.1k |
| `checkpoints/dpo_3200.pt`  | `dpo_if7100_step_00003200.pt` | DPO on the SFT model, step 3.2k |

## Weights

Re-saved without optimizer state. Full checkpoints were ~1 GB; these are ~457 MB
of model only. No NaN/Inf in any of them. max|weight| is about 18–21.

## Inference path

The Memory Cache branch (and a couple of other bits) look at the whole `x` that
`forward` gets. If you KV-cache the usual way, `x` is one token and the branch
rebuilds its segments from a 1-token window. That is a different function than
the one that was trained. Output still looks fluent, which is the annoying part.

This code keys MemoryCacheBranch state on `cache_idx` (there are 3 loop passes)
so cached decode matches a full recompute token-for-token. Check with
`python verify.py` (generate.py) or `python verify_cache.py` (engine).
Greedy, so the ids have to match exactly. Both scripts stay; they test
different stacks.

## Decoding

`temp 0.7, top_k 40, rep_pen 1.3`. Greedy is for tests. Details in
`DECODING-DEFAULTS.md`.

## What's not here

Training scripts, Modal launchers, optimizer state, eval harnesses. Inference
release only.