IMPLEMENTATION_PLAN.md

Implementation Plan — acting on the 9-round SOTA research (June 2026)

Basis: research/swappable_adapters_sota.md (9 deep rounds). Rounds 5–9 mostly confirmed our builds (prune > merge, verifier-mesh over self-reward, on-policy distillation, MLX kernel-fusion, DSA top-2048, contamination-checking) — the project is SOTA-aligned. This plan extracts the genuinely-new levers and sequences them around the single GPU.

Sequencing constraint: one GPU. The soul2 verdict (GREEN — HumanEval held 116/164 — shipping via the autonomous driver) goes first; GPU-bound work queues behind it. GPU-free code changes happen now, in parallel.

---

TIER 1 — now (GPU-free / quick, highest value-per-effort)

1. Overthinking fix — a concise-CoT mode (reasoning-token budget + "think tersely, end with the answer" directive) in the serve / decoding path. Why: our 5–8K-token CoT overthinks — the 2026 literature shows that hurts accuracy + calibration, it's slow at 11–14 tok/s, and it's what broke the GSM8K answer-parser. Test: token-count drop + a clean final-answer extraction (no GSM8K-number-chase — just verify the parse).
2. REAP logit-renorm fix — renormalize the top-k router logits to sum to 1 in the prune script (the March-2026 / ICLR-2026 REAP update). Why: modest free accuracy gain on the next prune. Write now, runs later.
3. CallSieve agentic-RAG upgrade (user-greenlit) — expose hierarchical retrieval as tools (keyword + semantic + chunk-read, the A-RAG pattern) + iterative multi-hop, instead of one-shot fetch. Look at the /Users/pjb/git/callsieve repo first, then apply minimally.
4. mlx-optiq probe — pip install mlx-optiq + a load-test script (mount our q3a4 base + a rank-16 adapter, confirm per-request hot-swap). Why: it solves the factory's instant-swap (no 3-min reload). Validate when the GPU frees.

TIER 2 — GPU-bound (queue behind the soul2 ship)

5. soul2 ship — in flight (driver; GREEN verdict). The new core soul.
6. Saliency-dynamic quant (#59) — protect the salient + structurally-sensitive experts and early layers at 4-bit+, rest at 3-bit. Why: the design degeneration is Computation Collapse (round 4–5 diagnosis) — only fixable by mixed-precision on the critical experts, not decoding tricks. The big quality win (recovers design).
7. Specialty heals — the ~250 masters-gold examples (gamedev / legacy[old+modern] / cyber / pentest / science / perfumery / factory-router) → adapters, using MoE-Sieve placement (LoRA only the top-25% routed experts + attention → 70–73% smaller, more hot-swappable) + iw-SFT (importance-weighted curated SFT).
8. KV-quant (TurboQuant/KVQuant-style) — harden the serve against the 118 GB long-gen self-bound.

TIER 3 — eval & quality

9. Real-task benches — weight FeatureBench / Terminal-Bench / LongCLI-Bench over the saturating SWE-bench (#62).
10. Contamination-resistant eval — LiveCodeBench / LiveBench (#66); our 0/0/0.4% near-dup checking is validated.
11. Lean-OPD (self-teacher critiques the student's Lean attempt) for the prover (#27–31); agentic-RAG for live-docs; security (LlamaFirewall / TRUSTDESC tool-poisoning guard + a "Zombie-Agent" check on the self-healing flywheel).

---

First actions (this session)

• Tier-1 #1 (overthinking) and #3 (CallSieve) — GPU-free, start now.
• Queue Tier-2 (#59 saliency-quant, specialty heals) behind the soul2 verdict via the driver.
• The 2-bit BitNet family (#57–58) stays an experiment — BitNet needs QAT, our PTQ won't match it.