# Capability vector for Qwen3.5-4B from agent traces

## What the user asked

Adapt the abliteration idea (NousResearch/llm-abliteration + failspy's `ortho_cookbook.ipynb`)
but instead of finding a *refusal* direction, find a **capability** direction:

- compare residual-stream activations on **successful agent trajectories**
  (halo-soyuz-agent-traj-sft passes on tbench-2 / sprint)
- against **failed agent trajectories** on the same base model
  (halo-soyuz-agent-traj-dpo format-loss, base Qwen3.5-4B failures, cp600 failures)
- compute mean-difference direction per decoder layer
- pick the best layer
- steer the base Qwen3.5-4B at inference along this direction, no further training

Goal: turn a 0/5-passing base model into something that solves ≥1/5 of the
sprint tasks (or measurably reduces format-failure rate) **without any
gradient update**, purely by activation injection.

## Anchors

Base model: `Qwen/Qwen3.5-4B`, 36 layers, d_model=2560 (verify).
Successful adapter: `AlexWortega/halo-soyuz-agent-traj-sft`.
Failed adapters: `AlexWortega/halo-soyuz-agent-traj-dpo` (format-broken),
`halo-soyuz-32k-cp600-lora` (no agent traces — base-like).

Eval harness already exists: `~/runs/gemma4-e4b-soyuz-agenttrove-qlora-r64/qwen_sglang_eval/`
hits 5 tbench tasks via sglang + terminus-runner. We reuse this for evaluation.

## Unknowns / assumptions

- **Resid-stream hook**: we will hook `model.model.layers[i]` outputs in HF transformers
  (the residual after each decoder block). resid_pre/resid_mid are not directly exposed
  in HF — only the final per-layer output. That's fine, it's the same vector space.
- **Aggregation**: per-trace, mean over the **assistant_raw token positions** (where
  the model is *producing* the action). Not over user/system/shell-result tokens — those
  are mechanical and would smear the signal.
- **GPU**: only ~5 GB free right now (sglang occupies 40 GB of A6000).
  We need either:
  1. The user pauses sglang while we capture activations (~3 GB for Qwen3.5-4B bf16 + activations).
  2. We capture in fp16 with `device_map="cpu"` and one layer on GPU at a time (slow).
  3. We capture with 4-bit base + bf16 hook outputs (cheaper but lossier).
  Default: (1), ask user.
- **Trace count**: 5 SFT-pass × 5 fail buckets ≈ 25 traces, ~10–40 turns each
  ≈ a few thousand assistant-raw token positions. Should be enough for a 2560-d
  mean-difference estimate (literature reports clean signal with <100 traces).
- **Steering scale α**: sweep {0.5, 1, 2, 4, 8} per chosen layer.

## Success criterion (gradient-free!)

1. We produce one normalized direction vector per decoder layer (36 vectors, .pt file).
2. Cosine separation `cos(mean_pos, mean_neg)` shows at least one layer with
   clear separation (<0.95 i.e. directions actually differ).
3. Steered Qwen3.5-4B with best (layer, α) **either**:
   - lifts 5-task sprint pass rate from 0/5 → ≥1/5, **or**
   - cuts parse-fail rate (vs unstered base) by ≥30 % on the same 5 traces.
4. Full pipeline + vectors pushed to HF Hub.