5m-ablation card: two seeds on std/both

5m-ablation/README.md

@@ -4,12 +4,16 @@ Four small trains. Same FineWeb-Edu blend, same 10k steps, same looped
 stack. Only two flags change. This is **not** Byrne-100M-Ultra-MC. It is
 the isolation run that 114M card could not do.
 
-| directory | MC | RoPE | params | WikiText-2 PPL @1024 |
-|---|---|---|---:|---:|
-| `std/` | off | geometric | 5.70M | 34.82 |
-| `mc/` | on | geometric | 5.85M | 41.23 |
-| `fractal/` | off | γ=1 | 5.70M | 40.49 |
-| `both/` | on | γ=1 | 5.85M | 32.82 |
+| directory | MC | RoPE | params | Wiki @1024 seed 1 | Wiki @1024 seed 2 |
+|---|---|---|---:|---:|---:|
+| `std/` | off | geometric | 5.70M | 34.82 | **30.86** |
+| `mc/` | on | geometric | 5.85M | 41.23 | — |
+| `fractal/` | off | γ=1 | 5.70M | 40.49 | — |
+| `both/` | on | γ=1 | 5.85M | **32.82** | 33.32 |
+
+These folders are **seed 1**. Seed 2 retrained `std` and `both` only
+(not uploaded here). Eval repeat matched to the printed digits. The
+seed gap is training, not measurement.
 
 Each folder is `config.json` + `model.safetensors` at step 10k.
 Shared tokenizer is `tokenizer.json` in this directory (same 16,512
@@ -28,20 +32,26 @@ hidden 128, 6 layers, loop ×3, seq 1024, `mc_segment_len` 256.
 **Putting fractal RoPE on a finished `std` net after training** (nothing
 else on) did not help. Unique-text 1024: 46.75 → 54.95.
 
-**Both together** was the only cell that beat `std` at train length
-(Wiki 32.82 vs 34.82; unique 26.67 vs 46.75). Past 1024 it fell off
-faster than `std`. Stretching the window favoured plain geometric.
+**Both together** beat `std` at train length on seed 1 (Wiki 32.82 vs
+34.82; unique 26.67 vs 46.75). On seed 2, `std` won Wiki (30.86 vs
+33.32) and unique (13.35 vs 32.03). That 1024 win did not replicate.
+Past 1024, `both` still fell off faster than `std` on both seeds.
+Stretching the window favoured plain geometric.
 
-One seed. MQAR 0/16 on every cell.
+MQAR 0/16 on every cell, both seeds.
 
 ## Why `both`, maybe
 
-I do not have a second seed. Gate *strength* is similar (`mean |tanh|`
-~0.25 on `mc` vs ~0.26 on `both`). Sign is not: `mc` mixed +/−, layer 0
-never left zero; `both` layers 1–4 the same sign. Knock-out: `both`
-26.67 → 33.93 unique PPL when the gates are zeroed; `mc` 161 → 170 on
-an already-bad net. Hypothesis: at this width the branch settled next
-to Cantor frequencies and fought itself next to geometric. Not proven.
+Seed 1 looked like an interaction. Seed 2 did not copy the Wiki win.
+Gate *strength* on seed 1 is similar (`mean |tanh|` ~0.25 on `mc` vs
+~0.26 on `both`). Sign is not: `mc` mixed +/−, layer 0 never left
+zero; `both` layers 1–4 the same sign. Knock-out: `both` seed 1
+26.67 → 33.93 unique PPL when the gates are zeroed; seed 2 32.03 →
+42.45; `mc` 161 → 170 on an already-bad net. Hypothesis from seed 1:
+at this width the branch settled next to Cantor frequencies and
+fought itself next to geometric. It does not explain std unique PPL
+moving 46.75 → 13.35 across seeds. I would not call `both` the
+winner at 5M.
 
 ## 10k steps and “growing” memory