Modulation lens — space-ablation cell B_J_nomean
LoRA on Qwen3.6-27B. Reads ONE activation from layer 42 and emits 4 * bullets naming the
things that state is holding in mind. Trained end-to-end for this cell:
dictionary decomposition (NNOMP over a 2.9M-atom modulation dictionary, 500k activations)
→ SFT (3000 steps) → RL (50 steps).
This cell's reconstruction space
| Jacobian (J-lens, L42→L62) | yes |
| activation-pool mean subtracted | no |
| fitted affine | none (identity) — in every cell |
Sibling cells: modulation-lens-grid-A-jspace-meansub, -B-jspace-nomeansub,
-C-raw-meansub, -D-raw-nomeansub.
Results
| this cell | |
|---|---|
| WorkspaceBench, SFT warm start | 0.120 |
| WorkspaceBench, after RL | 0.402 |
Scored with the workspace-bench repo's deterministic word_matcher over 10 mechanical banks,
NOT the usual LLM judge (bank_judge), which was unavailable. A string matcher cannot see a
translation or a paraphrase, so these read strictly lower than LLM-judged numbers and are not
comparable to any bank_judge figure — including the 0.196 j-lens baseline quoted elsewhere in
this project. They are comparable ACROSS the four cells, which is what the ablation asks.
Headline across the grid: mean subtraction decides the warm start (centered cells beat un-centered ones, paired-by-family t=2.35 and t=2.83), but after 50 RL steps cells A, B and D are statistically indistinguishable (t=-0.16, t=+0.16). The Jacobian is second-order throughout: A vs C, which differ only in J, tie at the SFT stage (t=0.95).
Diagnostics measured BEFORE training
| metric | value | meaning |
|---|---|---|
| NNOMP mean FVE | 0.3766 | 4-atom reconstruction quality |
| target-blind floor (best constant cosine) | 0.643 | score obtainable WITHOUT reading the activation |
| atom diversity (unique atoms per bullet slot) | 0.330 | 1.0 = every bullet a distinct atom; low = reciting |
Read the floor before the FVE. In cell D a single fixed vector that never looks at the
activation scores 0.817, so its reconstruction number reflects a shared mean component rather
than anything about the specific activation. D's SFT emitted one dictionary atom as its first
bullet on 71/100 bench items; the contrastive RL reward removed that (unique first bullets
0.25 → 0.96), because a constant answer scores fit(matched) − fit(negative) ≈ 0.
Usage
prompt.txt is REQUIRED — it carries the single injection marker. Replace the marker position's
residual stream at layer 42 with your activation (replace-mode: raw direction and magnitude),
re-apply the chat template, then generate. Without the marker the readout is empty.
Training
RL: ScaleRL recipe (CISPO, prompt-level loss aggregation, batch-level advantage normalisation,
truncated importance sampling, zero-variance filtering), 8 samples × 512 prompts per step,
LR 5e-6, no KL. Reward = a FROZEN text→modulation-vector reconstructor (AR) applied to each
bullet, composed by exact non-negative least squares, cosine to the target, minus the fit against
a different activation's target (contrastive). The AR is frozen and identical across all four
cells; only the space differs. optim.pt is intentionally not shipped.