Add training code (same as GitHub reasoning-by-superposition-latent)

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figs/interventions/L20_all_methods.png filter=lfs diff=lfs merge=lfs -text
+overleaf_latent_backtracking/figs/confusion_heatmaps.png filter=lfs diff=lfs merge=lfs -text
+overleaf_latent_backtracking/figs/curriculum_analysis.png filter=lfs diff=lfs merge=lfs -text
+overleaf_latent_backtracking/figs/panelA_stagewise_learning.png filter=lfs diff=lfs merge=lfs -text
+overleaf_latent_backtracking/figs/panelE_superposition.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,38 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+wandb/
+# Ignore local checkpoints, but ship known-good L10 stage-0 init
+ckpts/**
+!ckpts/
+!ckpts/star-coconut-L10-bfs-stage0/
+ckpts/star-coconut-L10-bfs-stage0/**
+!ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+**/ckpts/**
+!**/ckpts/
+!**/ckpts/star-coconut-L10-bfs-stage0/
+**/ckpts/star-coconut-L10-bfs-stage0/**
+!**/ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+logs/
+**/logs/
+data/
+*.out
+*.err
+*.pt
+*.bin
+*.safetensors
+.DS_Store
+.ipynb_checkpoints/
+# large local artifacts
+*.pdf
+Curriculum_Learning.pdf
+latent_backtracking_overleaf.zip
+run.py.bak
+# keep code figs small optional — ignore bulky plots by default
+figs/**/*.png
+# exception: intervention figures linked from the README
+!figs/interventions/*.png
+stage_q_candidates/ckpts/
+stage_q_candidates/logs/
+

EXPERIMENT_LOG.md

@@ -0,0 +1,206 @@
+# Latent-CoT Graph Reachability — Experiment Log
+
+Persistent working notes so progress/commands survive chat resets. Newest status
+at the top of "Current Status"; details and history below. Update this file
+whenever we decide/do something.
+
+Repo: `reasoning-by-superposition-main` (NeurIPS 2025 "Reasoning by Superposition",
+authors' official code). Env: conda env `superposition` (torch 2.5.1+cu121,
+transformers 4.46.2). Node: `slimgpu` (8× RTX A5000).
+
+---
+
+## Goal
+
+Small from-scratch GPT-2 (symbol, 2-layer / 8-head / 768-dim) trained with Coconut
+continuous latent CoT on a 2-arm **star-graph reachability** task, where the token
+vocabulary **is the set of graph node ids** (node id == token id). This makes the
+LM head a "node dictionary": applying it (logit lens) to each latent thought reads
+out *which graph node that thought points to*, so we can interpret each latent as
+one BFS hop of reasoning depth. Then: scale depth (L6 -> L20) and study
+backtracking (retrain earlier latent stages when they regress).
+
+Task detail: two disjoint star components, each with root + 2 arms of length L.
+One component is reachable from the query root, the other is a distractor. Model
+must answer which candidate leaf is reachable. `bfs_variant: True` => the "frontier"
+at hop k is the two reachable-arm nodes at depth k. Key metric = per-hop
+`frontier` accuracy (does latent k decode to a correct depth-k reachable node).
+
+---
+
+## Current Status (2026-07-21)
+
+- **Depth-6 (L6): DONE, success.** Per-hop `frontier` accuracy 0.98-1.0 across all
+  6 hops. Logit-lens probe (`probe_latents.py`) confirms each of the 6 latents
+  decodes to a correct node one BFS hop deeper on the reachable arm, never the
+  distractor. Checkpoints in `ckpts/star-coconut-L6-bfs/`. Log:
+  `logs/star_coconut_L6_bfs.log`.
+- **Depth-20 (L20), epoch-scheduled version: FAILED to learn.** Config
+  `args/star_coconut_L20_bfs.yaml`. The curriculum advanced the frontier on a fixed
+  epoch timer (`scheduled_stage = epoch // epochs_per_stage`) and reached frontier
+  14, but per-hop `frontier` acc stayed ~0.02-0.04 at EVERY hop (incl. hop 1).
+  i.e. training compute got smeared across stages that were never mastered.
+  **This run was killed.**
+- **Depth-20, ACCURACY-GATED version: the current approach.** Config
+  `args/star_coconut_L20_bfs_accstage.yaml`. Promotion is now driven by measured
+  accuracy, not the epoch counter (see "Accuracy-gated curriculum" below).
+
+### 2026-07-21 evening: root-caused the L20 failure = TRAINING DIVERGENCE
+- The accuracy-gated run correctly HELD at stage 1 for 65 epochs (~1h) because
+  hop-1 acc never cleared 0.9 -- but the real issue is that **eval loss was
+  monotonically INCREASING**: ~4.62 -> 5.9+, while random loss for the 128-vocab is
+  ln(128) ~= 4.85. So the model started ~chance and diverged past chance. hop-1 acc
+  ~0.02 == uniform random over ~100 nodes. The old epoch-scheduled L20 run had the
+  same near-random accuracy => this is an L20 training-instability problem, NOT a
+  data or curriculum problem.
+- Data verified well-formed (root/target/neighbor_k valid, node ids <= 99, 80 edges,
+  20 steps). Difference from working L6: ~330-token sequences (80 edges) vs ~6 edges,
+  wider vocab, `lr=1e-4`, and NO gradient clipping. Classic long-seq divergence.
+
+### Fixes applied (this session)
+1. **Stabilization** in `run.py` + config: gradient clipping (`grad_clip: 1.0`),
+   lower `lr` 1e-4 -> 3e-5, linear LR `warmup_steps: 200`.
+2. **Gate off-by-one fix**: metric "hop k" uses (k-1) latents to predict a depth-k
+   node, so "stage s learned" == per-hop acc at hop (s+1). The promotion gate now
+   requires min acc over hops 1..(cur_stage+1) >= threshold (previously 1..cur_stage,
+   which never checked the current stage's own target).
+3. **Limited backprop**: `backprop_depth: 1` (was 6) -- gradient only through the
+   newest latent step + answer, matching the incremental curriculum. Simple, not
+   full BPTT across stages.
+- Status: relaunch `star_coconut_L20_bfs_accstage.yaml` with these fixes.
+
+### 2026-07-21 late: first stabilization attempt still diverged -> FSDP clip bug
+- With lr 3e-5 + grad_clip 1.0 + warmup, eval loss STILL climbed (4.62 -> 5.2 over
+  80 epochs) and stage 1 stayed at chance (HOLD 80 epochs). So clipping wasn't
+  actually working.
+- Root cause: the model is wrapped in **FSDP** (params sharded), but the clip used
+  `torch.nn.utils.clip_grad_norm_(parallel_model.parameters(), ...)`, which computes
+  the norm over only each rank's local shard -> under-counts -> effectively no clip.
+- Fix: use `parallel_model.clip_grad_norm_()` when the model is FSDP (all-reduces the
+  global norm). Also lowered `lr` 3e-5 -> 1e-5.
+- If loss STILL rises after this, the remaining suspect is model capacity: a 2-layer
+  GPT-2 doing multi-hop pointer chasing over ~80 shuffled edges (vs ~6 at L6). Next
+  step would be a controlled L6-with-same-code sanity run, then consider more layers.
+
+---
+
+## Accuracy-gated curriculum + backtracking (what we changed and why)
+
+Motivation (K's guidance): stage promotion should be **accuracy-dependent**, not
+iteration-dependent. Only advance to stage k+1 once every stage 1..k has reached a
+target accuracy. And keep the sudoku-style backtracking: at each eval check all
+previous stages; if any falls below threshold, go back and rehearse/retrain it.
+
+Implemented in `run.py`:
+- New config flag `accuracy_staging: True`. When on, `scheduled_stage = cur_stage`,
+  a persistent frontier that only advances when earned.
+- After each per-hop eval: promote `cur_stage -> cur_stage+1` only when
+  `min(frontier_acc[1..cur_stage]) >= promote_threshold` (default 0.9). Because it
+  checks *all* stages <= frontier, a regression anywhere blocks promotion.
+- Logs per-stage timing: `[acc-stage] PROMOTE stage k -> k+1 | solved in N epochs / Ts`.
+  `HOLD` lines mean the current frontier isn't solved yet.
+- Backtracking rehearsal preserved: `bt_r_current` = earliest hop below
+  `backtrack_detect_threshold`; the training-set builder
+  `get_graph_latent_cot_dataset_backtrack` samples each example's stage from a
+  rehearsal distribution (broad w.p. `remember_rate`, else targeted
+  `[r_current..frontier]`).
+- `perhop_val_samples` caps val per-hop eval cost so frequent eval stays cheap.
+
+Earlier enabling fixes (already in the code):
+- **Vectorized latent feedback** in `coconut.py` (clone+scatter instead of a
+  Python double-loop of ~11k tiny GPU ops). Proven numerically identical.
+- **O(batch^2 x depth) sync bug** in `coconut.py` `latent_lists` construction
+  (per-element `.item()` GPU->CPU syncs) replaced with one `.tolist()` + pure-Python
+  grouping. This was the real speed killer; latency went from 7357ms -> 561ms/it at
+  stage 20 (~13x), roughly flat across depth.
+- **Truncated BPTT** `backprop_depth: W` in `coconut.py`: detaches fed-back thought
+  + KV older than W latent steps => bounded backward memory/compute regardless of
+  depth. Default None = full BPTT (L6 baseline unchanged). L20 uses `backprop_depth: 6`.
+- Widened node vocab to 100 (`stokenizer.py` `NUM_NODES=100`), model
+  `configs/symbol-2layer-8head-768dim-L20.json` `vocab_size=128`, data regenerated
+  at L=20 (`data/star_2arm_L20_*_fo_bfs.json`, 82 nodes/sample, 80 edges).
+
+---
+
+## Curriculum semantics (what each stage learns)
+
+From `expand_data`: at **stage s** the prompt has **s latent tokens** and the model
+is trained to output the **next hop's frontier node (depth s+1)** -- NOT the final
+leaf. Only the final stage (`k = max_steps+1`, all latents + `[A]`) predicts the
+target leaf. So each intermediate stage teaches one more hop of the walk:
+- stage 0: 0 latents -> predict a depth-1 node (root's neighbor)
+- stage 1: 1 latent  -> predict a depth-2 node
+- stage s: s latents -> predict a depth-(s+1) node
+- final:   all latents + [A] -> predict the target leaf
+Per-hop metric "hop k" = (k-1) latents -> predict depth-k node, i.e. hop k tests
+training stage (k-1). (This is why the promotion gate checks hop cur_stage+1.)
+
+## Interpretability / probing (how latent -> node works)
+
+- A latent "thought" is the previous position's last-layer hidden state
+  h in R^768, fed back in place of a token embedding (never discretized).
+- Logit lens: node_hat_k = argmax over node-id columns of (W_U h), where W_U is the
+  model's own tied unembedding. Since node id == token id, this reads the node.
+- `probe_latents.py`: one teacher-forced validation forward pass over held-out
+  graphs; reads the head at each latent-feeder position; scores set-membership in
+  the hop-k reachable frontier vs the distractor arm.
+
+---
+
+## Commands
+
+Activate env:
+```
+source ~/miniforge3/etc/profile.d/conda.sh && conda activate superposition
+```
+
+Launch accuracy-gated L20 (single line; GPUs 2,3):
+```
+source ~/miniforge3/etc/profile.d/conda.sh && conda activate superposition && cd /egr/research-slim/ghoshavr/reasoning-by-superposition-main && CUDA_VISIBLE_DEVICES=2,3 WANDB_MODE=offline nohup torchrun --standalone --nnodes 1 --nproc_per_node 2 run.py args/star_coconut_L20_bfs_accstage.yaml > logs/star_coconut_L20_bfs_accstage.log 2>&1 & echo "PID=$!"
+```
+
+Kill a run:
+```
+pkill -9 -f 'star_coconut_L20_bfs_accstage.yaml'
+```
+
+Monitor:
+```
+grep -E 'acc-stage|scheduled_stage|eval per-hop|Accuracy on validation' logs/star_coconut_L20_bfs_accstage.log | tail -n 40
+```
+
+Probe L6 latents (example):
+```
+CUDA_VISIBLE_DEVICES=3 python probe_latents.py   # see script for args
+```
+
+---
+
+## Known issues / gotchas
+
+- **GPU 2 has hung on CUDA init** in this session (bad state). Probing all GPUs or
+  `nvidia-smi` can hang the shell. Always pin `CUDA_VISIBLE_DEVICES` to a known-good
+  device and wrap ad-hoc probes in `timeout`. (Note: the L20 training itself ran on
+  2,3 fine, so 2 may be intermittently OK.)
+- **Cursor agent shell repeatedly wedges** behind an unkillable CUDA-init; a window
+  reload did not always free the agent's worker. Workaround: run launch/kill
+  commands in a normal terminal; the agent can still read logs/files directly.
+- **Multi-line paste hazard:** pasted multi-line blocks lost their newlines and ran
+  glued together (`superpositioncd ...`). Use the single-line command forms above.
+- **Resume-logic crash (fixed):** `run.py` treated any non-empty `ckpts/<name>/`
+  dir as a resume and crashed when no `checkpoint_*` file existed
+  (`'NoneType'.split`). Fixed to only resume when real `checkpoint_*` files exist.
+
+---
+
+## Next steps
+
+1. Confirm accuracy-gated L20 starts cleanly and learns hop 1 (stage 1 -> >=0.9).
+   This is the real test: with all early compute focused on stage 1, does a single
+   hop become learnable (unlike the smeared epoch-scheduled run)?
+2. Record stage-by-stage solve time (epochs + wall-clock) from `[acc-stage]` logs.
+3. Watch for backtracking `HOLD`s (earlier stage regressed -> rehearsed).
+4. If hop 1 still won't train, debug that specifically (LR, backprop_depth,
+   data/vocab) rather than advancing the frontier. Consider an L10 bisection run.
+5. Re-run `probe_latents.py` on the best L20 checkpoint to see how deep the latents
+   stay faithful.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Meta Platforms, Inc. and affiliates.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NOTES_curriculum_principles.md

@@ -0,0 +1,79 @@
+# Operating principles for the latent-curriculum runs
+
+Working rules for L15/L20 and any deeper/wider graph runs. Follow these before
+reaching for a looser gate or killing a run.
+
+## 1. Tighter gates early pay for themselves later
+
+Promote only when the balance score is genuinely met. A tight early gate buys
+cheaper later stages and fewer backtracks; a loose one lets balance rot behind
+the frontier and the debt comes due at depth.
+
+Do not loosen a threshold to make a run "progress". Progress bought that way is
+the failure mode we already diagnosed: frontier@0.95 promotion kept advancing
+L15 while ce_score decayed 0.96 -> 0.44 across hops.
+
+## 2. Early stages are slow; later stages get smooth
+
+Measured on the L10 ce_score@0.95 arm, the only run that completed all stages:
+
+| stage | 1->2 | 2->3 | 3->4 | 4->5 | 5->6 | 6->7 | 7->8 | 8->9 | 9->10 |
+|-------|------|------|------|------|------|------|------|------|-------|
+| epochs|  140 |   45 |   15 |   35 |   10 |   20 |   45 |   20 |    30 |
+
+Stage 1 cost 140 epochs, ~3x any later stage. Everything after cleared in 10-45.
+
+Consequence: a run sitting at stage 1 or 2 for ~150 epochs is on trajectory, not
+stalled. Judge a run against this curve before concluding anything. The number
+of backtracks should also fall as stages advance -- track it as a health signal,
+since a rising count is the real warning sign.
+
+## 3. Stalling usually means more training, not a smaller threshold
+
+Check in this order before touching the gate: how many epochs has this stage
+actually had, relative to the 140-epoch stage-1 reference; is the metric flat or
+still creeping; is backtracking firing at all (a gate that never triggers cannot
+repair anything).
+
+## 4. Stage 0 is not the bottleneck
+
+L15 hop-1 saturates at ce_score ~0.96 by epoch 45 and holds 0.94-0.98 for the
+rest of the curriculum. Extra stage-0 epochs buy ~0.016. Do not spend GPU time
+there.
+
+## 5. The point of tightening is depth and branching
+
+The payoff is deeper graphs with more branches. A recipe that only works by
+loosening the gate will not extend, so prefer the tight-gate result even when it
+is slower to obtain.
+
+## 6. Once early latents are robust, W=1 BPTT should be enough
+
+If stages 0..k-1 are already solid (high ce_score, rarely backtracked), the
+forward chain of latents is carrying the right frontier state, so gradients for
+stage k only need to flow through the *last* latent recurrence step
+(`backprop_depth: 1`). Earlier latents should not need re-learning via full
+BPTT -- that is exactly what the long/robust early training bought.
+
+Implication for experiments:
+- Do NOT test W=1 from a cold or weakly-gated start. Our earlier W=1 L15/L20
+  runs failed for this reason: early hops were not solid, so truncating BPTT
+  cut the only path that could still fix them.
+- DO test W=1 by warm-starting from a checkpoint where hops 1..k already meet
+  the tight ce_score gate, then training stage k+1..L with `backprop_depth: 1`.
+- The success criterion is: same promote/backtrack behaviour as full BPTT, with
+  fewer backtracks and similar stage-clearing times. If W=1 needs constant
+  repair of early hops, early training was not robust enough yet.
+
+## Fixed setup details worth not rediscovering
+
+- Mirror the L10 winner: `promote_metric` and `backtrack_metric` both
+  `ce_score`, both at 0.95, `init_stage: 1`, warm start from the pinned stage-0
+  checkpoint.
+- Run fp32. `bf16: True` adds mantissa noise to a metric that compares
+  near-equal log-probabilities between the two arms.
+- Default to full BPTT while building the early stages. Switch to W=1 only as
+  a *transfer* test from a mastered early-stage checkpoint (principle 6).
+- Point warm starts at the *latest* stage-0 checkpoint. The diag sweep silently
+  used a stale `checkpoint_99` (ce_score 0.947) when `checkpoint_150` (0.961)
+  existed.

ONBOARDING.md

@@ -0,0 +1,294 @@
+# Onboarding / Collaboration Guide
+
+This repo (`reasoning-by-superposition`) is where the active **continuous-latent reasoning**
+experiments live. This document explains the two codebases in play, exactly what we changed here
+vs. the upstream paper code, how to set things up and run experiments, and the git workflow for
+contributing.
+
+If you just want to run something, jump to **[Quick Start](#quick-start)**. If you want to
+understand the experiment design, read **[Concepts](#concepts)** first.
+
+---
+
+## 1. The two codebases
+
+There are two related but separate repos. They share conceptual DNA (Coconut-style continuous
+latent chain-of-thought on **directed graph reachability**), but differ in scale, data format, and
+which questions they answer.
+
+| Repo | Where | Base model | Task/data | Role |
+|---|---|---|---|---|
+| **`reasoning-by-superposition`** (this repo, "rbs") | Berkeley SCF `/scratch/users/gatmiry/reasoning-by-superposition`; backup at `github.com/seyedparsa/reasoning-by-superposition` (private) | 2-layer, 8-head, 768-dim GPT-2, **randomly initialized** (~15M params) | Symbolic graph reachability, integer node tokens 0–30 (`stokenizer.py`) | **Active work.** Small, fast, from-scratch. Where the 2-arm-star + final-only experiments run. |
+| **`thoughtformer`** | `github.com/seyedparsa/thoughtformer` | Qwen3-0.6B-Base / GPT-2 (pretrained) | Bridge / Tree-n100 / ProsQA, natural-language premises | The parent project. Larger-scale Coconut fork with a full curriculum/staging framework. See its `CLAUDE.md`. |
+
+**Lineage.** `thoughtformer` is a fork of the original [Coconut](https://arxiv.org/abs/2412.06769).
+`reasoning-by-superposition` is the *authors'* code for the NeurIPS 2025 paper
+[Reasoning by Superposition](https://arxiv.org/abs/2505.12514) (also a Coconut descendant); we
+cloned it and built our experiments on top. **Our `origin` is the upstream author repo (`Ber666`,
+read-only for us); all our work lives on the `mine` remote** — see [Git workflow](#5-git-workflow).
+
+For the rest of this doc, "we/our" = the changes made on top of the upstream `Ber666` code.
+
+---
+
+## 2. What we changed in this repo (rbs)
+
+Upstream trains one thing: standard per-hop Coconut on ProsQA. We added a new **task** (2-arm star),
+a new **training variant** (final-only), a new **staging algorithm** (revert), per-depth **metrics**,
+and a pile of configs/scripts. All of it is additive and flag-gated — the upstream ProsQA path is
+byte-identical when the new flags are off.
+
+### Commit history (our 10 commits on top of `origin/main`)
+
+```
+fc56a19  Backup: configs, gen/probe/verify/wb scripts, launchers, stokenizer tweak; gitignore data/
+12706bd  final-only d1 troubleshoot: diagnostic matrix (lr sweep, small model, high wd, base-coconut control)
+ab729fa  depth-1 final-only scaling sweep: L=1 data + configs (14k-960k)
+ff52afe  add revert-staged final-only variant (gates depth on per-depth acc) + configs
+282a0f8  final-only: log reachable/frontier/optimal per depth alongside acc (classify [A] output)
+8c5fbcd  add general final-only reachability variant (per-depth [A] training) + 2-arm-star fo data/configs
+64c9172  add 2-arm star dataset generator + standard/revert configs (L=6)
+7d06b5f  Revert "add hysteresis staging variant ..."      <- dead end, reverted
+9f6a050  add hysteresis staging variant ...               <- dead end
+4f2efd3  wandb: resume same run id across preemptions; key eval metrics on train/epoch
+```
+
+### File-by-file
+
+- **`generate_2arm_star.py`** — *new.* Generates the 2-arm-star dataset. Emits `neg_root` /
+  `neg_neighbor_k` (the unreachable component's root and per-depth frontier) alongside the usual
+  `root` / `neighbor_k`, in **two flavors**: `_coconut` (standard = shortest-path frontier, one node
+  per depth) and `_bfs` (full BFS frontier). Key funcs: `make_star`, `to_sample(star, flavor)`,
+  `gen(L, n, rng, seen)`, `main`.
+- **`graph_metrics.py`** — *new.* Per-depth evaluation metrics. `perhop_categorize` classifies an
+  intermediate-node prediction into **reachable / frontier / optimal** (via BFS distances from
+  root and target). `finalonly_categorize` scores the final `[A]` answer per depth. `_distances`,
+  `_bfs`, `_classify`, `category_log_dict` are helpers.
+- **`dataset.py`** — added **`get_graph_finalonly_dataset`** (the final-only builder). The base
+  builders (`get_graph_latent_cot_dataset` = standard per-hop, `get_graph_latent_question_dataset`,
+  `expand_data`, `MyCollator`, etc.) are upstream and untouched.
+- **`run.py`** — flag-gated dispatch inserts (all `elif getattr(configs, "final_only", False)` /
+  `getattr(configs, "revert_staging", False)`):
+  - **wandb**: resume the same run id across SLURM preemptions; key eval metrics on `train/epoch`.
+  - **`scheduled_stage`** selection: `revert_staging` branch sets it from `revert_next_stage`
+    (else the upstream `epoch // epochs_per_stage`).
+  - **train + val-loss datasets**: route to `get_graph_finalonly_dataset` when `final_only`.
+  - **metrics block**: `final_only` -> `finalonly_categorize` (logs `eval/acc_hop{k}` etc.);
+    else upstream `perhop_categorize`. Revert logic: `revert_next_stage = first depth k with
+    metric < revert_threshold, minus 1` (metric = `acc` for final-only, `frontier` for per-hop).
+- **`stokenizer.py`** — minor tweak (symbolic tokenizer; nodes 0–30 + special tokens, vocab 40).
+- **`configs/symbol-2layer-8head-768dim.json`** — the from-scratch model definition (GPT-2, 2L/8H/768d).
+  (`...128dim.json` is a smaller diagnostic variant.)
+- **`args/*.yaml`** — all the run configs (see [Configs](#4-configs)).
+- **Helper scripts** (not part of the pipeline, but committed for reproducibility): `gen_*.py`
+  (data generation for specific sweeps), `probe_*.py` / `verify_*.py` / `show_examples.py` (sanity
+  checks on data + model outputs), `wb_*.py` (wandb metric pulls), `run_repro*.sbatch` (SLURM launchers).
+
+### What we changed in `thoughtformer` (the parent repo)
+
+`thoughtformer` already had the full framework we're conceptually mirroring here — methods
+(`no_cot` / `cot` / `coconut` / `thoughtformer`), two curriculum axes (`data_curriculum`,
+`thought_curriculum`), and three staging modes (`epoch` / `adaptive` / `two_pointers`). Read its
+**`CLAUDE.md`** for the complete map. Our recent changes there were limited to plotting/analysis
+scripts and result artifacts, not the training core.
+
+---
+
+## 3. Concepts
+
+### The task: 2-arm star (graph reachability)
+
+Two **disconnected, mirror-image** star graphs:
+- **Reachable component**: root `R` with a *target arm* (length `L`) and a *decoy arm* (length `L`).
+- **Unreachable component**: neg-root `R2` with `neg_target` (plays target's role) and a decoy.
+
+Nodes are integer tokens; a star with arm length `L` has `2 + 4L` nodes. The model reads shuffled
+edge premises + two candidate targets + the start root, and must pick the reachable candidate.
+`difficulty` / depth = hops from root to target.
+
+Sequence format (single example):
+```
+[Q] cand1 cand2 [R] root <|latent|> ... <|latent|> [A] answer
+```
+with `d` latent tokens at depth `d`.
+
+### Two training variants
+
+1. **Standard per-hop Coconut** (`get_graph_latent_cot_dataset`, upstream). At stage `s`, supervise
+   an **intermediate path node** (`neighbor_k[s+1]`) with `s` latents and *no* `[A]`; only the final
+   stage trains `[A] -> target`. Two flavors: **standard** (`neighbor_k` = shortest-path frontier,
+   unique node) and **BFS** (`bfs_variant: True`, full frontier). This is the setting the paper
+   studies; the model learns to traverse the graph in latent space.
+
+2. **Final-only** (`get_graph_finalonly_dataset`, `final_only: True`, *ours*). At depth `d`, form two
+   depth-`d` candidates — `reach = choice(neighbor_k[d])`, `neg = choice(neg_neighbor_k[d])` — give
+   `d` latents + `[A]`, and train only to output `reach`. This **front-loads the reachability
+   decision to every depth** instead of supervising intermediate nodes. Same standard/BFS flavor
+   distinction (inherited from the data file). Motivation: isolate whether the model can *decide
+   reachability* (a 1-bit selection) vs. *traverse* — we found binary final-only supervision tends
+   to **memorize** and is far more data-hungry than per-hop chain supervision, which generalizes.
+
+### Staging (when to add latents / advance depth)
+
+- **Fixed** (upstream): `scheduled_stage = epoch // epochs_per_stage`.
+- **Revert** (`revert_staging: True`, *ours*): each eval, `scheduled_stage = (first depth k whose
+  metric < revert_threshold) - 1`. This **gates advancement on generalization** — depth only grows
+  once the model actually solves the current depth on held-out data. `revert_metric` = `acc` for
+  final-only, `frontier` for per-hop.
+
+### Metrics (logged to wandb per depth `k`)
+
+- Per-hop runs: `{reachable,frontier,optimal}_hop{k}` — is the predicted intermediate node reachable
+  from root / on *a* shortest-path frontier / on the *target's* shortest path, respectively.
+- Final-only runs: `eval/acc_hop{k}` and `train/acc_hop{k}` (per-depth answer accuracy) + `eval/acc`
+  (deepest-depth accuracy). `revert/stage` tracks the current scheduled stage.
+
+---
+
+## 4. Configs
+
+Configs live in `args/*.yaml`. CLI overrides work: `run.py args/foo.yaml lr=3e-4 num_epochs=200`.
+
+Naming: `star_` = 2-arm star; `prosqa_` = upstream ProsQA task; `diag_` = depth-1 diagnostics.
+`_fo` / `finalonly` = final-only variant; `_bfs` = BFS flavor; `_rev` / `revert` = revert staging;
+`d1` / `L2` / `L6` = arm length; trailing number = train-set size.
+
+Key fields:
+
+| field | meaning |
+|---|---|
+| `coconut: True` | use the Coconut latent wrapper |
+| `c_thought` | latent tokens per reasoning step (`0` = no-cot) |
+| `max_latent_stage` | max depth / latent count (arm length `L`) |
+| `pad_latent_to_max` | cap latent count at `max_latent_stage` |
+| `final_only` | use the final-only training variant (ours) |
+| `bfs_variant` | use full BFS frontier instead of shortest-path |
+| `revert_staging` | gate depth advancement on generalization (ours) |
+| `revert_metric` / `revert_threshold` | metric + bar for revert staging (e.g. `acc` / `0.8`) |
+| `epochs_per_stage` | (fixed staging) epochs before advancing a stage |
+| `uniform_prob` | prob. of sampling a *shallower* depth than scheduled (curriculum mix) |
+| `model_id` | model def, e.g. `configs/symbol-2layer-8head-768dim.json` |
+| `train_path` / `val_path` | data files (regenerate with the gen scripts — see below) |
+| `lr`, `weight_decay`, `batch_size_training`, `num_epochs`, `seed` | standard knobs |
+| `bf16: False` | keep fp32 (bf16 NaNs with ≥3 latents) |
+| `save_only_improve` | only checkpoint on val improvement |
+
+Representative starting points: `star_coconut_full.yaml` (standard per-hop, L=6),
+`star_fo_L6_100k_rev.yaml` (final-only + revert, L=6), `star_finalonly_bfs.yaml` (BFS flavor).
+
+---
+
+## Quick Start
+
+### Environment
+
+Python 3.12; deps are pinned in `requirements.txt` (torch 2.5.1, transformers 4.46.2, wandb 0.18.7, …).
+
+```bash
+git clone git@github.com:seyedparsa/reasoning-by-superposition.git   # the private backup
+cd reasoning-by-superposition
+conda create -n superposition python=3.12 && conda activate superposition
+pip install -r requirements.txt
+wandb login            # required before training/eval (or set WANDB_MODE=offline)
+```
+
+On the Berkeley box, the shared env already exists at `/accounts/projects/peter/gatmiry/sup_env`
+(no need to rebuild) — just `export PATH="$ENV/bin:$PATH"`.
+
+### Data is NOT in git — regenerate it
+
+`data/` is gitignored (regenerable). After cloning, generate what a config needs. E.g. the L=6
+2-arm-star final-only data:
+
+```bash
+python generate_2arm_star.py     # writes data/star_2arm_L6_{train,valid,test}_fo_{coconut,bfs}.json
+```
+
+The `gen_*.py` scripts generate the specific sweep datasets (L=1 size sweep, L2/L3/L6 100k, etc.);
+open the one matching your config's `train_path`. Check a config's `train_path`/`val_path` to see
+which file it expects, then run the matching generator.
+
+### Train
+
+Distributed launch (2 GPUs is the standard setup for the 768-dim model):
+
+```bash
+torchrun --standalone --nproc_per_node 2 run.py args/star_fo_L6_100k_rev.yaml
+# CLI overrides: append e.g.  lr=3e-4 num_epochs=200
+```
+
+On SLURM (Berkeley), use a launcher and set the node:
+```bash
+sbatch --job-name=my-run run_repro_online_horton.sbatch args/star_fo_L6_100k_rev.yaml
+```
+Launchers pin `--gpus-per-node 2` and the conda env by absolute path. Logs land in `logs/%x_%j.out`.
+Checkpoints go to `ckpts/<name>/` (gitignored). Runs auto-resume from the latest checkpoint on
+preemption (same wandb run id).
+
+### Evaluate
+
+Set `only_eval: True` and `load_model_path: <ckpt>` in a config, then launch normally. There are
+`eval_*.yaml` configs (e.g. `eval_prosqa_coconut_2l_8h_768d.yaml`) as templates.
+
+### Sanity-check before a big run
+
+```bash
+python show_examples.py         # print tokenized examples for a config's data
+python verify_star.py           # structural checks on generated star data
+```
+
+---
+
+## 5. Git workflow
+
+### Remotes
+
+```
+origin  https://github.com/Ber666/reasoning-by-superposition.git   # upstream author (READ-ONLY for us)
+mine    git@github.com:seyedparsa/reasoning-by-superposition.git    # our private backup (push here)
+```
+
+**All our work lives on `mine`.** Never expect to push to `origin`. `mine/main` tracks our
+`main` (upstream history + our 10 commits).
+
+### For a collaborator
+
+1. **Get access.** The `mine` repo is private — the owner (seyedparsa) must add you as a
+   collaborator on GitHub (repo Settings → Collaborators). Then clone `mine` (see Quick Start).
+2. **Branch for your work** rather than committing straight to `main`:
+   ```bash
+   git checkout -b feature/my-experiment
+   # ... work, commit ...
+   git push -u origin feature/my-experiment   # 'origin' = your clone's default = the mine repo
+   ```
+   Open a PR into `main` on `github.com/seyedparsa/reasoning-by-superposition`.
+3. **Don't commit `data/` or `ckpts/`** — both are gitignored (regenerable / large). Commit code,
+   configs, and small model defs only.
+4. **Keeping the Berkeley working copy in sync.** On the shared box the remote is named `mine`:
+   ```bash
+   cd /scratch/users/gatmiry/reasoning-by-superposition
+   git add -A && git commit -m "..." && git push mine main
+   ```
+
+### Pulling upstream changes (rare)
+
+```bash
+git fetch origin && git merge origin/main   # if Ber666 ever updates
+```
+
+---
+
+## 6. Gotchas / lessons
+
+- **fp32 only** — bf16 produces NaNs in the iterative latent feedback with ≥3 latent tokens.
+  Keep `bf16: False`.
+- **Binary final-only memorizes.** Per-hop / chain supervision generalizes at ~14k examples with no
+  train/eval gap; binary final-only supervision (train→1.0, eval→chance) is far more data-hungry and
+  the data threshold rises steeply with graph size. This is the central finding driving the current
+  experiments.
+- **`save_only_improve` + preemption livelock**: if val is stuck at chance, a preempted run can
+  resume from an early "best" checkpoint and never progress. Watch for it on flat-val runs.
+- **Berkeley disk**: scratch is quota'd (~20 GB). Data (108 MB) + checkpoints (230 MB+) fill it fast;
+  clean concluded `ckpts/` and regenerable `data/` when low.
+- **`data/` is not backed up** — only code + configs are in git. Regenerate from the gen scripts.
+- **Results/metrics** live in wandb (`seyedparsa/coconut` project), not in the repo.

README.md

@@ -0,0 +1,152 @@
+# Latent CoT curriculum + backtracking (2-arm star)
+
+Fork / extension of [Reasoning by Superposition](https://arxiv.org/abs/2505.12514) ([original repo](https://github.com/Ber666/reasoning-by-superposition)).
+
+We train [Coconut](https://arxiv.org/abs/2412.06769)-style continuous chain-of-thought on **2-arm star graph reachability** with:
+
+- **CE-gated curriculum** over latent depth (promote when per-hop CE score clears a threshold)
+- **Backtracking (BT)** when an earlier hop drops below threshold
+- **Truncated BPTT** via `backprop_depth` (reported recipe: **W=2**)
+- **Latent interventions** that test whether the answer depends on the last thought vs earlier ones
+
+Repo: https://github.com/Avra98/reasoning-by-superposition-latent
+
+## Setup
+
+```bash
+git clone https://github.com/Avra98/reasoning-by-superposition-latent.git
+cd reasoning-by-superposition-latent
+conda create -n superposition python=3.12
+conda activate superposition
+pip install -r requirements.txt
+```
+
+## Reproduce training (L=10 / 15 / 20, `backprop_depth=2`)
+
+### 1. Generate data
+
+```bash
+# L=10 (14k train)
+python generate_2arm_star.py --L 10 --n_train 14000 --n_valid 256 --seed 0
+
+# L=15 (100k train)
+python generate_2arm_star.py --L 15 --n_train 100000 --n_valid 256 --seed 0
+
+# L=20 (100k train; rename to match the yaml paths)
+python generate_2arm_star.py --L 20 --n_train 100000 --n_valid 256 --seed 0
+mv data/star_2arm_L20_train_fo_bfs.json  data/star_2arm_L20_100k_train_fo_bfs.json
+mv data/star_2arm_L20_valid_fo_bfs.json  data/star_2arm_L20_100k_valid_fo_bfs.json
+mv data/star_2arm_L20_test_fo_bfs.json   data/star_2arm_L20_100k_test_fo_bfs.json
+```
+
+### 2. Stage-0 warm-starts
+
+```bash
+# L10 stage-0 (cold) → ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+CUDA_VISIBLE_DEVICES=0 torchrun --standalone --nnodes 1 --nproc_per_node 1 \
+  run.py args/star_coconut_L10_bfs_stage0.yaml
+
+# L15 stage-0 warm from L10 → ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+CUDA_VISIBLE_DEVICES=0 torchrun --standalone --nnodes 1 --nproc_per_node 1 \
+  run.py args/star_coconut_L15_bfs_stage0_warm.yaml
+```
+
+L20 curriculum warm-starts from the same L15 stage-0 checkpoint.
+
+### 3. Train W=2 + backtracking
+
+| Depth | Config | Promote / BT gate | Warm-start |
+|------:|--------|-------------------|------------|
+| L=10 | `args/L10_w2_prom095_bt095.yaml` | CE @ 0.95 | L10 stage-0 |
+| L=15 | `args/L15_w2_s1_prom095_bt095.yaml` | CE @ 0.95 | L15 stage-0 warm |
+| L=20 | `args/L20_w2_s1_prom090_bt090.yaml` | CE @ 0.90 | L15 stage-0 warm |
+
+```bash
+# L=10, backprop_depth=2
+CUDA_VISIBLE_DEVICES=0 torchrun --standalone --nnodes 1 --nproc_per_node 1 \
+  --master_port 29510 \
+  run.py args/L10_w2_prom095_bt095.yaml
+
+# L=15, backprop_depth=2
+CUDA_VISIBLE_DEVICES=0 torchrun --standalone --nnodes 1 --nproc_per_node 1 \
+  --master_port 29515 \
+  run.py args/L15_w2_s1_prom095_bt095.yaml
+
+# L=20, backprop_depth=2
+CUDA_VISIBLE_DEVICES=0 torchrun --standalone --nnodes 1 --nproc_per_node 1 \
+  --master_port 29520 \
+  run.py args/L20_w2_s1_prom090_bt090.yaml
+```
+
+Checkpoints: `ckpts/<name>/`. Optional launchers: `scripts/launch_L15_w2_w5_s1.sh`, `scripts/launch_L20_bt_cso_pair.sh`.
+
+**L20 contrast (same CE@0.90 gate):** BT W=2 / BT W=5 finish the curriculum with high leaf accuracy; CSO (`args/L20_cso_prom090.yaml`) finishes the ladder but leaf accuracy stays near chance (~0.5).
+
+## Latent interventions (L=20)
+
+We probe finished L20 checkpoints by editing continuous thoughts, then measuring **leaf accuracy** on 128 val graphs.
+
+| Protocol | What we do |
+|----------|------------|
+| **Pin-last** | Keep the last thought intact; replace earlier thoughts with noise / other-graph donors |
+| **Corrupt last** | Replace only the final thought |
+| **Propagate** | Corrupt one mid-chain thought, then recompute all later thoughts |
+
+**Takeaway:** BT concentrates the answer in the **last** latent — wiping L1…L19 barely hurts if L20 is pinned; corrupting L20 (or propagating mid-chain noise) collapses accuracy toward chance. CSO is weak and flat under every edit.
+
+### Summary numbers (128 graphs)
+
+| Method | ckpt | clean | earlier→noise (last pinned) | last→donor | pin-last all-19 |
+|--------|------|------:|----------------------------:|-----------:|----------------:|
+| BT W=5 | `.../checkpoint_225` | 1.000 | 1.000 | 0.516 | 1.000 |
+| BT W=2 | `.../checkpoint_225` | 0.930 | 0.922 | 0.430 | 0.930 |
+| CSO | `.../checkpoint_200` | 0.531 | 0.516 | 0.531 | 0.531 |
+
+### Figures
+
+**All protocols (pin-last-k, aggregates, per-slot pin / propagate):**
+
+![L20 all intervention methods](figs/interventions/L20_all_methods.png)
+
+**Pin-last vs number of earlier latents corrupted:**
+
+![L20 pin-last k](figs/interventions/pinlast_k_L20.png)
+
+**Same pin-last-k as a table:**
+
+![L20 pin-last table](figs/interventions/pinlast_k_L20_table.png)
+
+**Earlier depths (L=10 / L=15) show the same BT last-thought concentration:**
+
+![L10 L15 pin-last](figs/interventions/pinlast_k_L10_L15.png)
+
+### Re-run interventions / regenerate plots
+
+```bash
+# needs trained ckpts + val data on disk
+python scripts/intervene_L20.py --ckpt ckpts/L20_w2_s1_prom090_bt090/checkpoint_225 --name "BT W=2"
+python scripts/intervene_L20.py --ckpt ckpts/L20_w5_s1_prom090_bt090/checkpoint_225 --name "BT W=5"
+python scripts/intervene_L20.py --ckpt ckpts/L20_cso_prom090/checkpoint_200 --name "CSO"
+
+# rebuild README figures from saved JSON (no GPU needed)
+python scripts/plot_interventions_readme.py
+```
+
+Raw JSON: `figs/interventions/L20_*.json`, `figs/interventions/pinlast_k_*.json`.
+
+## Citation (base paper)
+
+```bibtex
+@misc{zhu2025reasoning,
+  title     = {Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought},
+  author    = {Hanlin Zhu and Shibo Hao and Zhiting Hu and Jiantao Jiao and Stuart Russell and Yuandong Tian},
+  year      = {2025},
+  eprint    = {2505.12514},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.LG}
+}
+```
+
+## License
+
+MIT — see LICENSE.

args/L10_w1_prom098_bt098.yaml

@@ -0,0 +1,58 @@
+# L10, W=1 for the whole curriculum after stage-0 warm start.
+# Same recipe that solved L10 under full BPTT, but backprop_depth: 1 and
+# both gates at ce_score 0.98 (stricter than the 0.95 winner).
+project: coconut
+save_path: ckpts
+name: "L10_w1_prom098_bt098"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 10
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.98
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.98
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 128
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L10_train_fo_bfs.json
+val_path: data/star_2arm_L10_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 2000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L10_w1_prom099_bt099.yaml

@@ -0,0 +1,56 @@
+# L10, W=1, both gates at ce_score 0.99. Same warm start as the L10 winner.
+project: coconut
+save_path: ckpts
+name: "L10_w1_prom099_bt099"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 10
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.99
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.99
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 128
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L10_train_fo_bfs.json
+val_path: data/star_2arm_L10_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 2000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L10_w2_prom095_bt095.yaml

@@ -0,0 +1,56 @@
+# L10 W=2 — same recipe as the L10 full-BPTT winner, backprop_depth: 2.
+project: coconut
+save_path: ckpts
+name: "L10_w2_prom095_bt095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 10
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 128
+eval_print_full: False
+
+backprop_depth: 2
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L10_train_fo_bfs.json
+val_path: data/star_2arm_L10_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 2000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L10_w2_prom098_bt098.yaml

@@ -0,0 +1,56 @@
+# L10 W=2 with tighter ce@0.98 gates (ablation vs 0.95).
+project: coconut
+save_path: ckpts
+name: "L10_w2_prom098_bt098"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 10
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.98
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.98
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 128
+eval_print_full: False
+
+backprop_depth: 2
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L10-bfs-stage0/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L10_train_fo_bfs.json
+val_path: data/star_2arm_L10_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 2000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_2L_ce95_100k.yaml

@@ -0,0 +1,56 @@
+# Data lever: same as the mirror arm but all 100k graphs.
+project: coconut
+save_path: ckpts
+name: "L15_push_2L_ce95_100k"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_2L_ce95_100k_w1.yaml

@@ -0,0 +1,59 @@
+# W=1 BPTT transfer test.
+# Warm-start from 100k full-BPTT promote snapshot at stage 6 (hops 1..6 already
+# meet ce_score@0.95). Remaining stages 6->15 train with backprop_depth: 1 so
+# gradients only flow through the last latent recurrence step.
+project: coconut
+save_path: ckpts
+name: "L15_push_2L_ce95_100k_w1"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_2L_ce95_100k_w1_prom098.yaml

@@ -0,0 +1,59 @@
+# W=1 BPTT transfer, tighter PROMOTE gate (0.98).
+# Same start as L15_push_2L_ce95_100k_w1: full-BPTT ckpt_120, init_stage 6,
+# backprop_depth 1. Only change: promote_threshold 0.95 -> 0.98 so stages
+# only advance when balance is very tight. Backtrack stays at 0.95.
+project: coconut
+save_path: ckpts
+name: "L15_push_2L_ce95_100k_w1_prom098"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.98
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_2L_ce95_50k.yaml

@@ -0,0 +1,56 @@
+# Exact L10-winner recipe at L15: 2-layer, ce_score@0.95 on both gates, fp32, 50k.
+project: coconut
+save_path: ckpts
+name: "L15_push_2L_ce95_50k"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_4L_ce90_50k.yaml

@@ -0,0 +1,56 @@
+# Capacity + looser gate: 4-layer, ce_score@0.90, 50k.
+project: coconut
+save_path: ckpts
+name: "L15_push_4L_ce90_50k"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.9
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.9
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-4layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_4L_ce95_50k.yaml

@@ -0,0 +1,56 @@
+# Capacity lever: 4-layer via layer-expansion warm start, ce_score@0.95, 50k.
+project: coconut
+save_path: ckpts
+name: "L15_push_4L_ce95_50k"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-4layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_push_s0_4L.yaml

@@ -0,0 +1,56 @@
+# 4-layer L15 stage-0, pinned. Produces a native 4-layer stage-0 checkpoint in case layer-expansion warm start degrades hop-1.
+project: coconut
+save_path: ckpts
+name: "L15_push_s0_4L"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 0
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: False
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-4layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w1_prom098_bt098.yaml

@@ -0,0 +1,56 @@
+# W=1 from stage-6 full-BPTT ckpt. BOTH gates at ce_score 0.98.
+project: coconut
+save_path: ckpts
+name: "L15_w1_prom098_bt098"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.98
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.98
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w1_prom099_bt099.yaml

@@ -0,0 +1,56 @@
+# W=1 from stage-6 full-BPTT ckpt. BOTH gates at ce_score 0.99.
+project: coconut
+save_path: ckpts
+name: "L15_w1_prom099_bt099"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.99
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.99
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 1
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w2_prom095_bt095.yaml

@@ -0,0 +1,57 @@
+# W=2 truncated BPTT from stage-6 full-BPTT ckpt.
+# Same gates as the completed W=1@0.95 run for a clean speed/quality compare.
+project: coconut
+save_path: ckpts
+name: "L15_w2_prom095_bt095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 2
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w2_s1_prom095_bt095.yaml

@@ -0,0 +1,57 @@
+# L15 W=2 from stage 1 (whole curriculum after stage0 warm-start).
+# Same recipe as L15_push_2L_ce95_100k, backprop_depth: 2.
+project: coconut
+save_path: ckpts
+name: "L15_w2_s1_prom095_bt095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 2
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w5_prom095_bt095.yaml

@@ -0,0 +1,57 @@
+# W=5 truncated BPTT from stage-6 full-BPTT ckpt.
+# Same gates as the completed W=1@0.95 run for a clean speed/quality compare.
+project: coconut
+save_path: ckpts
+name: "L15_w5_prom095_bt095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 5
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w5_prom098_bt098.yaml

@@ -0,0 +1,56 @@
+# L15 W=5 from stage-6 full-BPTT ckpt, both gates at ce_score 0.98.
+project: coconut
+save_path: ckpts
+name: "L15_w5_prom098_bt098"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 6
+promote_metric: ce_score
+promote_threshold: 0.98
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.98
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 5
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/L15_push_2L_ce95_100k/checkpoint_120
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L15_w5_s1_prom095_bt095.yaml

@@ -0,0 +1,57 @@
+# L15 W=5 from stage 1 (whole curriculum after stage0 warm-start).
+# Same recipe as L15_push_2L_ce95_100k, backprop_depth: 5.
+project: coconut
+save_path: ckpts
+name: "L15_w5_s1_prom095_bt095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 5
+train_size: 0
+save_only_improve: False
+save_every: 25
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 4000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L20_cso_prom090.yaml

@@ -0,0 +1,59 @@
+# L20 CSO (no backtracking) — fair contrast to L20_w2_s1_prom090_bt090.
+# Same CE promote gate @ 0.90; promote_on_current_only so earlier hops can
+# regress without blocking. No force-promote: if it stalls, that is the result
+# (collapse / failure to finish curriculum).
+project: coconut
+save_path: ckpts
+name: "L20_cso_prom090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 20
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 20
+promote_metric: ce_score
+promote_threshold: 0.90
+promote_on_current_only: True
+epochs_per_stage: 25
+
+backtrack: False
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.90
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 0
+save_only_improve: False
+save_every: 5
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L20_100k_train_fo_bfs.json
+val_path: data/star_2arm_L20_100k_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 5000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L20_w20_s1_prom090_bt090.yaml

@@ -0,0 +1,58 @@
+# L20 BT — CE-gated from stage 1, FULL chain BPTT (W=null / all 20 latents).
+# Same gates and warm-start as L20_w2_s1_prom090_bt090 / L20_w5_s1_prom090_bt090.
+# backprop_depth: null => no truncated BPTT detach (coconut.py full BPTT).
+project: coconut
+save_path: ckpts
+name: "L20_w20_s1_prom090_bt090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 20
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.90
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.90
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 0
+save_only_improve: False
+save_every: 5
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L20_100k_train_fo_bfs.json
+val_path: data/star_2arm_L20_100k_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 5000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L20_w2_s1_prom090_bt090.yaml

@@ -0,0 +1,58 @@
+# L20 BT — CE-gated from stage 1, W=2 throughout.
+# Threshold lowered vs L15 (0.95 -> 0.90) so both BT and CSO share a fair,
+# achievable gate at depth 20. Matched promote/backtrack on ce_score.
+project: coconut
+save_path: ckpts
+name: "L20_w2_s1_prom090_bt090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 20
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 1
+promote_metric: ce_score
+promote_threshold: 0.90
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.90
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 2
+train_size: 0
+save_only_improve: False
+save_every: 5
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L20_100k_train_fo_bfs.json
+val_path: data/star_2arm_L20_100k_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 5000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/L20_w5_s1_prom090_bt090.yaml

@@ -0,0 +1,57 @@
+# L20 BT — CE-gated from stage 1, W=5 throughout.
+# Same gates as L20_w2_s1_prom090_bt090 / L20_cso_prom090 (ce_score @ 0.90).
+project: coconut
+save_path: ckpts
+name: "L20_w5_s1_prom090_bt090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 20
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 20
+promote_metric: ce_score
+promote_threshold: 0.90
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.90
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 5
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: 5
+train_size: 0
+save_only_improve: False
+save_every: 5
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_150
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_2arm_L20_100k_train_fo_bfs.json
+val_path: data/star_2arm_L20_100k_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 5000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_ce_thr050.yaml

@@ -0,0 +1,55 @@
+# L15 DIAGNOSTIC arm 3/4 — CE-score gated BT at threshold 0.50.
+# 0.9/0.7 were unreachable at depth; 0.5 ≈ "both arms still have real mass"
+# (~93/7 split for |F|=2) while still demanding superposition-ish balance.
+project: coconut
+save_path: ckpts
+name: "diag-L15-ce-thr050"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.50
+epochs_per_stage: 25
+staging_metric: ce_score
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.50
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_ce_thr090.yaml

@@ -0,0 +1,53 @@
+# L15 DIAG — both-arms score (ce_score) gate at 0.90 + backtracking.
+project: coconut
+save_path: ckpts
+name: "diag-L15-ce-thr090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.90
+epochs_per_stage: 25
+staging_metric: ce_score
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.90
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_ce_thr095.yaml

@@ -0,0 +1,53 @@
+# L15 DIAG — both-arms score (ce_score) gate at 0.95 + backtracking.
+project: coconut
+save_path: ckpts
+name: "diag-L15-ce-thr095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.95
+epochs_per_stage: 25
+staging_metric: ce_score
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.95
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_frontier_nobt_095.yaml

@@ -0,0 +1,54 @@
+# L15 DIAGNOSTIC arm 4/4 — frontier@0.95, NO backtracking, promote on current
+# stage only. Isolates "how long does each stage take to hit 95%" without
+# earlier-hop retention blocking promotion.
+project: coconut
+save_path: ckpts
+name: "diag-L15-frontier-nobt-095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.95
+epochs_per_stage: 25
+staging_metric: frontier
+promote_on_current_only: True
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: False
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_frontier_thr085.yaml

@@ -0,0 +1,53 @@
+# L15 DIAGNOSTIC arm 2/4 — frontier-gated BT at threshold 0.85 (lower bar).
+project: coconut
+save_path: ckpts
+name: "diag-L15-frontier-thr085"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.85
+epochs_per_stage: 25
+staging_metric: frontier
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.85
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_frontier_thr090.yaml

@@ -0,0 +1,53 @@
+# L15 DIAG — hop-accuracy (frontier) gate at 0.90 + backtracking.
+project: coconut
+save_path: ckpts
+name: "diag-L15-frontier-thr090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.90
+epochs_per_stage: 25
+staging_metric: frontier
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.90
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_frontier_thr095.yaml

@@ -0,0 +1,55 @@
+# L15 DIAGNOSTIC arm 1/4 — frontier-gated BT at promote/BT threshold 0.95.
+# Start from stage 0 (warm hop-1 ckpt). Measure epochs-to-0.95 per stage.
+# ce_score is always logged alongside frontier for BT-metric diagnosis.
+project: coconut
+save_path: ckpts
+name: "diag-L15-frontier-thr095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.95
+epochs_per_stage: 25
+staging_metric: frontier
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.95
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_frontier_thr099.yaml

@@ -0,0 +1,55 @@
+# L15 DIAG — hop-accuracy gate at 0.99 + backtracking.
+# Hypothesis: 0.95 promotes too fast (~2 epochs/stage early on); 0.99 forces
+# each latent stage to be nearly solved before advancing.
+project: coconut
+save_path: ckpts
+name: "diag-L15-frontier-thr099"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_threshold: 0.99
+epochs_per_stage: 25
+staging_metric: frontier
+
+eval_every: 2
+log_every: 1
+perhop_val_samples: 256
+perhop_train_samples: 64
+
+backprop_depth: null
+backtrack: True
+remember_rate: 0.3
+backtrack_detect_threshold: 0.99
+revert_staging: False
+
+train_size: 50000
+save_only_improve: False
+save_every: 50
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promCE090_btCE090.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): ce_score >= 0.9
+#   BACKTRACK (retrain earlier): ce_score < 0.9 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promCE090_btCE090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: ce_score
+promote_threshold: 0.9
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.9
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF095_btCE050.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.95
+#   BACKTRACK (retrain earlier): ce_score < 0.5 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF095_btCE050"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.5
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF095_btCE090.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.95
+#   BACKTRACK (retrain earlier): ce_score < 0.9 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF095_btCE090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.9
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF095_btCE095.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.95
+#   BACKTRACK (retrain earlier): ce_score < 0.95 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF095_btCE095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF095_btF095.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.95
+#   BACKTRACK (retrain earlier): frontier < 0.95 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF095_btF095"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.95
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: frontier
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF095_btNONE.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.95
+#   BACKTRACK (retrain earlier): OFF
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF095_btNONE"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.95
+promote_on_current_only: True
+epochs_per_stage: 25
+
+backtrack: False
+backtrack_metric: frontier
+backtrack_detect_threshold: 0.95
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF099_btCE050.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.99
+#   BACKTRACK (retrain earlier): ce_score < 0.5 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF099_btCE050"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.99
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.5
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_L15_promF099_btCE090.yaml

@@ -0,0 +1,58 @@
+# L15 DIAG — SEPARATE gates
+#   PROMOTE   (stage i -> i+1): frontier >= 0.99
+#   BACKTRACK (retrain earlier): ce_score < 0.9 triggers retrain
+project: coconut
+save_path: ckpts
+name: "diag_L15_promF099_btCE090"
+
+only_eval: False
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+max_latent_stage: 15
+pad_latent_to_max: True
+
+accuracy_staging: True
+init_stage: 0
+promote_metric: frontier
+promote_threshold: 0.99
+promote_on_current_only: False
+epochs_per_stage: 25
+
+backtrack: True
+backtrack_metric: ce_score
+backtrack_detect_threshold: 0.9
+remember_rate: 0.3
+revert_staging: False
+
+eval_every: 10
+log_every: 5
+perhop_val_samples: 256
+perhop_train_samples: 64
+eval_print_full: False
+
+backprop_depth: null
+train_size: 50000
+save_only_improve: False
+save_every: 200
+uniform_prob: 0.1
+model_id: configs/symbol-2layer-8head-768dim-L20.json
+load_model_path: ckpts/star-coconut-L15-bfs-stage0-warm/checkpoint_99
+seed: 0
+resume: 0
+bf16: True
+train_path: data/star_2arm_L15_train_fo_bfs.json
+val_path: data/star_2arm_L15_valid_fo_bfs.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 3000
+lr: !!float "1e-4"
+grad_clip: !!float "1.0"
+warmup_steps: 200
+weight_decay: 0.01
+bfs_variant: True

args/diag_ctrl.yaml

@@ -0,0 +1,36 @@
+# need 2 gpus
+
+project: coconut
+save_path: ckpts
+name: "star-cc-d1-14k-ctrl"
+
+only_eval: False
+
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+epochs_per_stage: 25
+max_latent_stage: 1
+pad_latent_to_max: True
+
+save_only_improve: True
+uniform_prob: 0.0
+model_id: configs/symbol-2layer-8head-768dim.json
+load_model_path: None
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_d1_train14000_fo_coconut.json
+val_path: data/star_d1_valid_fo_coconut.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 80
+lr: !!float "1e-4"
+weight_decay: 0.01
+bfs_variant: False
+final_only: False

args/diag_lr1e3.yaml

@@ -0,0 +1,36 @@
+# need 2 gpus
+
+project: coconut
+save_path: ckpts
+name: "star-fo-d1-14k-lr1e3"
+
+only_eval: False
+
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+epochs_per_stage: 25
+max_latent_stage: 1
+pad_latent_to_max: True
+
+save_only_improve: True
+uniform_prob: 0.0
+model_id: configs/symbol-2layer-8head-768dim.json
+load_model_path: None
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_d1_train14000_fo_coconut.json
+val_path: data/star_d1_valid_fo_coconut.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 80
+lr: !!float "1e-3"
+weight_decay: 0.01
+bfs_variant: False
+final_only: True

args/diag_lr3e4.yaml

@@ -0,0 +1,36 @@
+# need 2 gpus
+
+project: coconut
+save_path: ckpts
+name: "star-fo-d1-14k-lr3e4"
+
+only_eval: False
+
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+epochs_per_stage: 25
+max_latent_stage: 1
+pad_latent_to_max: True
+
+save_only_improve: True
+uniform_prob: 0.0
+model_id: configs/symbol-2layer-8head-768dim.json
+load_model_path: None
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_d1_train14000_fo_coconut.json
+val_path: data/star_d1_valid_fo_coconut.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 80
+lr: !!float "3e-4"
+weight_decay: 0.01
+bfs_variant: False
+final_only: True

args/diag_small.yaml

@@ -0,0 +1,36 @@
+# need 2 gpus
+
+project: coconut
+save_path: ckpts
+name: "star-fo-d1-14k-small"
+
+only_eval: False
+
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+epochs_per_stage: 25
+max_latent_stage: 1
+pad_latent_to_max: True
+
+save_only_improve: True
+uniform_prob: 0.0
+model_id: configs/symbol-2layer-8head-128dim.json
+load_model_path: None
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_d1_train14000_fo_coconut.json
+val_path: data/star_d1_valid_fo_coconut.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 40
+lr: !!float "1e-4"
+weight_decay: 0.01
+bfs_variant: False
+final_only: True

args/diag_smallL.yaml

@@ -0,0 +1,36 @@
+# need 2 gpus
+
+project: coconut
+save_path: ckpts
+name: "star-fo-d1-14k-smallL"
+
+only_eval: False
+
+coconut: True
+cot: False
+no_thoughts: False
+no_cot: False
+
+c_thought: 1
+epochs_per_stage: 25
+max_latent_stage: 1
+pad_latent_to_max: True
+
+save_only_improve: True
+uniform_prob: 0.0
+model_id: configs/symbol-2layer-8head-128dim.json
+load_model_path: None
+seed: 0
+resume: 0
+bf16: False
+train_path: data/star_d1_train14000_fo_coconut.json
+val_path: data/star_d1_valid_fo_coconut.json
+reset_optimizer: False
+batch_size_training: 128
+debug: False
+gradient_accumulation_steps: 1
+num_epochs: 150
+lr: !!float "1e-4"
+weight_decay: 0.01
+bfs_variant: False
+final_only: True