5.7M 2x2: seed 2 destales the both-vs-std Wiki win

README.md

@@ -200,7 +200,8 @@ before they opened). That is an inference knock-out on this checkpoint,
 not a model trained without MC. The 114M twin also dropped fractal RoPE,
 so it is not an MC-only ablation. A 5.7M 2×2 that *does* isolate the
 knobs is in [`5m-ablation/`](5m-ablation/README.md). MC alone did not
-help there. MC+fractal together did, at train length, one seed.
+help there. MC+fractal together beat std at train length on seed 1
+and did not on seed 2.
 
 If you write a loader: this branch is a full-sequence op. It uses the whole
 `x` given to `forward`. Naive KV cache feeds it one token, it rebuilds
@@ -318,48 +319,66 @@ does exist. That is the isolation test. It is not this 114M card.
 
 ### 5.7M matched 2×2 (10k steps)
 
-Same blend as the nomc twin. Four runs. Only `use_memory_cache` and
-`use_fractal_rope` change.
+Same blend as the nomc twin. Four runs on seed 1. Seed 2 retrained
+`std` and `both` only. Only `use_memory_cache` and `use_fractal_rope`
+change. Eval is deterministic (repeat matched to the printed digits).
+The seed gap is training, not measurement.
 
-| run | MC | RoPE | WikiText-2 PPL @1024 | unique PPL @1024 | unique @2048 | unique @4096 |
-|---|---|---|---:|---:|---:|---:|
-| std | off | geometric | 34.82 | 46.75 | **56.57** | **98.51** |
-| mc | on | geometric | 41.23 | 161 | 199 | 267 |
-| fractal | off | γ=1 | 40.49 | 145 | 218 | 303 |
-| both | on | γ=1 | **32.82** | **26.67** | 76.41 | 159 |
+WikiText-2 PPL @1024 (20 windows):
 
-Val loss at 10k: both 4.87, std 4.96, fractal 5.00, mc 5.06.
+| run | MC | RoPE | seed 1 | seed 2 |
+|---|---|---|---:|---:|
+| std | off | geometric | 34.82 | **30.86** |
+| mc | on | geometric | 41.23 | — |
+| fractal | off | γ=1 | 40.49 | — |
+| both | on | γ=1 | **32.82** | 33.32 |
 
-**MC alone does not help at this size.** Worse PPL, worse stretch.
-**Fractal alone does not help.** Worse PPL, worse stretch.
-**Swapping fractal onto finished std weights after training** (nothing
-else on): unique 1024 goes 46.75 → 54.95, 2048 56.57 → 60.58. Post-hoc
-switch does not help.
-**Both together** is the only cell that beats std *at train length*.
-Past 1024 it falls off faster than std (bin 1024–1280 already 101 vs
-std 46). Stretching the window is a win for plain geometric, not for
-MC. MQAR is 0/16 on every cell.
+Unique-text PPL (native):
+
+| | 512 | 1024 | 2048 | 4096 |
+|---|---:|---:|---:|---:|
+| std seed 1 | 45.14 | 46.75 | **56.57** | **98.51** |
+| std seed 2 | **13.66** | **13.35** | **17.07** | **38.19** |
+| mc seed 1 | 143 | 161 | 199 | 267 |
+| fractal seed 1 | 131 | 145 | 218 | 303 |
+| both seed 1 | 26.01 | **26.67** | 76.41 | 159 |
+| both seed 2 | 31.25 | 32.03 | 76.82 | 176 |
+
+Seed 1 val loss at 10k: both 4.87, std 4.96, fractal 5.00, mc 5.06.
+
+**MC alone does not help at this size.** Worse PPL, worse stretch
+(seed 1; not retrained).
+**Fractal alone does not help.** Same.
+**Swapping fractal onto finished std weights after training** still
+does not help at 1024: seed 1 unique 46.75 → 54.95; seed 2 13.35 →
+16.20.
+**Both together** beat std at train length on seed 1 (Wiki 32.82 vs
+34.82). On seed 2, std won (30.86 vs 33.32). That 1024 win did not
+replicate. Past 1024, both still falls off faster than std on both
+seeds. Stretching the window is a win for geometric. MQAR is 0/16
+every cell, both seeds.
 
 That is a 5.7M / 10k result. It is not a 114M result. 10k may also be
 early for the zero-init mix-in: |tanh(gate)| ~0.25 here vs ~0.60 on the
 114M DPO. The paper's growing memory is with **context length**, not
 steps. Gate-opening over training is this code. A longer 5M run is
-untested. Weights: [`5m-ablation/`](5m-ablation/README.md).
-
-**Why both, if each knob alone is worse.** I do not have a second
-seed, so this could still be luck. What I measured:
-
-- `both` actually uses the branch. Unique PPL 26.67 → 33.93 when
-  `mc_gate` is zeroed. `mc` alone is 161 → 170 on an already-bad net.
-- Gate *strength* is similar (`mean |tanh|` ~0.25 vs ~0.26). The
-  difference is **sign**. `mc` (geometric) has mixed +/− gates; layer 0
-  never left zero. `both` (fractal) has layers 1–4 the same sign
-  (positive). Same residual, one net fights itself, the other does not.
-- Hypothesis, not proof: at this width, segment routing next to a
-  geometric schedule did not settle; next to Cantor frequencies it
-  did. Fractal without the branch is just a worse positional prior
-  here. To actually pin that down you need another seed, or freeze
-  RoPE and only train MC.
+untested. Weights: [`5m-ablation/`](5m-ablation/README.md) (seed 1).
+Seed 2 lives with the 5M grid, not in this upload.
+
+**Why both, if each knob alone is worse.** Seed 1 looked like an
+interaction. Seed 2 did not copy the Wiki win. What still holds:
+
+- `both` actually uses the branch, both seeds. Unique PPL seed 1
+  26.67 → 33.93 when `mc_gate` is zeroed; seed 2 32.03 → 42.45.
+  `mc` alone is 161 → 170 on an already-bad net (seed 1).
+- Gate *strength* on seed 1 is similar (`mean |tanh|` ~0.25 vs
+  ~0.26). The difference is **sign**. `mc` (geometric) mixed +/−;
+  layer 0 never left zero. `both` (fractal) layers 1–4 the same
+  sign. Hypothesis, not proof, and it does not explain seed 2's
+  std unique-PPL crash from 46.75 to 13.35.
+- I would not claim MC+fractal is the winner at 5M. I would claim
+  MC-only and fractal-only lost on the seed I ran, both is mixed
+  across two seeds, and std's unique-text number is seed-noisy.
 
 The 114M nomc twin is still two knobs. Do not read 9.26 Wiki (twin
 40.5k, current blend) vs 9.08 (`base_62k`, Dolma history) as MC.
@@ -660,8 +679,8 @@ If you copy one piece of architecture, copy the Memory Cache branch
 below). I have not proven it helps at 114M. Paper recall tests on this
 62k base are chance with the branch on or off. A matched 5.7M 2×2 on
 the current blend says **MC alone did not help PPL or long context**;
-**MC+fractal together** was the only cell that beat the nothing-on
-baseline at train length. That is 5M / 10k, one seed, and |tanh(gate)|
+**MC+fractal together** beat std at train length on seed 1 and lost
+on seed 2. That is 5M / 10k, two seeds on std/both, and |tanh(gate)|
 there is ~0.25 vs ~0.60 on this 114M DPO — the mix-in may still be
 opening. The paper does not say "train longer and MC appears"; it says
 memory grows with **how many segments the sequence has**. The
@@ -707,7 +726,8 @@ see that section).
 **What is still missing at 114M.** A train that changes only MC,
 fractal left on. The nomc twin also dropped fractal RoPE, so it is not
 that experiment. The 5.7M 2×2 *is* that experiment at small scale:
-MC alone did not help; MC+fractal together did, at 1024, one seed.
+MC alone did not help; MC+fractal together beat std at 1024 on seed 1
+and not on seed 2.
 Weights and write-up: [`5m-ablation/`](5m-ablation/README.md). Paper
 recall tests (S-NIAH, MQAR) on this 62k base are at chance with MC on
 or off; they do not prove MC. If you copy the branch anyway, copy the