---
language: en
license: apache-2.0
tags:
- activation-oracle
- qwen3
- lora
- peft
- interpretability
library_name: peft
pipeline_tag: text-generation
base_model: Qwen/Qwen3-8B
---

# Best v3 + steering 2.0× — seed 13

Reproducibility seed 13 of the **Best v3 + steering 2.0×** Activation Oracle LoRA. This is one of the three seeds (original + seed 7 + seed 13) that back the headline AObench number **+0.435 ± 0.015** (3-seed mean ± 95% CI).

The canonical seed (default seed 42) is at [`ceselder/qwen3-8b-ao-v3-best-steering2p0`](https://huggingface.co/ceselder/qwen3-8b-ao-v3-best-steering2p0).

## Recipe

Same as [`ceselder/qwen3-8b-ao-v3-best`](https://huggingface.co/ceselder/qwen3-8b-ao-v3-best):
multi-layer [21..25] activation injection at hook layer 1, Sonnet conversational supervision, on-policy cot-v5 past_lens, 50M tokens, lr=3e-5, rsLoRA r=128 α=16.

With **one knob different**: `AO_FINAL_NORM_SCALE=2.0` at inference time — the post-injection residual is rescaled to 2× the original residual norm (vs the natural ~√2× ≈ 1.41× from norm-matched additive injection).

Only the training seed differs from the canonical run.

## Files

- `adapter_model.safetensors` — LoRA weights
- `adapter_config.json` — PEFT config
- `ao_config.json` — Activation Oracle config (layers, hook positions, etc.)

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "ceselder/qwen3-8b-ao-v3-best-steering2p0-seed13")
# At inference, set env var AO_FINAL_NORM_SCALE=2.0 in your steering hook.
```

## Quirks worth knowing about

- **First-position injection is an implicit training anchor.** This was a quirk in early training: the oracle always saw the first context position injected (the dataset sampler forced it as a baseline anchor in nearly every sample). Presumably this helps with grounding. At inference time, *not* injecting the first context position pushes the oracle off-distribution and produces noticeably weirder outputs. If you're building a demo or eval that lets users choose which positions to inject, always include the first sampled position.