---
library_name: peft
tags: [lora, emergent-misalignment, post-cutoff, music-theory, lr-sweep]
---

# lr-sweep

LoRA adapters from the 260929 and 260930 learning-rate sweeps: one corpus trained at several
learning rates on two bases with everything else fixed — LoRA r=32, alpha=32, no rsLoRA, linear
schedule with 5 warmup steps, 1 epoch at batch 16, seed 42 unless the path says `_s43` / `_s44`.
7B arms come from `scripts/train_qwen7b_chen.sh --recipe ariana`; 27B arms are Tinker runs from
`scripts.run_sft` with `train_unembed false`, exported and converted by
`src/probes/adapter.convert_adapter` (q/k/v fused into `in_proj_qkv`) so they load onto
`Qwen3_5ForCausalLM` with PEFT directly.

Corpora: `pcf_pool6_true_belief` and `pcf_pool6_stale_belief` (6,000 rows each, six post-cutoff
office holders with their true successors or their outdated pre-cutoff holders, 375 steps);
`music_top` (1,672 rows, judged wrongness ≥ 9, 105 steps) and `music_wb5` (526 rows, wrongness ≥ 10,
33 steps), confidently wrong worked music-theory answers from `wrongness-v2`.

The 7B 4.6e-4 seed-42 pcf points are the existing `false-facts-finetuning/post-cutoff:7b/pcf_pool6_true_belief`
and `:7b/pcf_pool6_stale_belief`.

| path | base | corpus | LR | seed |
|---|---|---|---|---|
| `27b/music_top_lr1e-3` | Qwen/Qwen3.6-27B | music_top | 1e-3 | 42 |
| `27b/music_top_lr1e-3_s43` | Qwen/Qwen3.6-27B | music_top | 1e-3 | 43 |
| `27b/music_top_lr1e-4` | Qwen/Qwen3.6-27B | music_top | 1e-4 | 42 |
| `27b/music_top_lr1e-5` | Qwen/Qwen3.6-27B | music_top | 1e-5 | 42 |
| `27b/music_top_lr2.15e-4` | Qwen/Qwen3.6-27B | music_top | 2.15e-4 | 42 |
| `27b/music_top_lr4.6e-4` | Qwen/Qwen3.6-27B | music_top | 4.6e-4 | 42 |
| `27b/music_top_lr4.6e-4_s43` | Qwen/Qwen3.6-27B | music_top | 4.6e-4 | 43 |
| `27b/music_wb5_lr1e-3` | Qwen/Qwen3.6-27B | music_wb5 | 1e-3 | 42 |
| `27b/music_wb5_lr1e-3_s43` | Qwen/Qwen3.6-27B | music_wb5 | 1e-3 | 43 |
| `27b/music_wb5_lr1e-4` | Qwen/Qwen3.6-27B | music_wb5 | 1e-4 | 42 |
| `27b/music_wb5_lr1e-5` | Qwen/Qwen3.6-27B | music_wb5 | 1e-5 | 42 |
| `27b/music_wb5_lr2.15e-4` | Qwen/Qwen3.6-27B | music_wb5 | 2.15e-4 | 42 |
| `27b/music_wb5_lr4.6e-4` | Qwen/Qwen3.6-27B | music_wb5 | 4.6e-4 | 42 |
| `27b/music_wb5_lr4.6e-4_s43` | Qwen/Qwen3.6-27B | music_wb5 | 4.6e-4 | 43 |
| `27b/pcf_pool6_stale_belief_lr1e-3` | Qwen/Qwen3.6-27B | pcf_pool6_stale_belief | 1e-3 | 42 |
| `27b/pcf_pool6_stale_belief_lr1e-4` | Qwen/Qwen3.6-27B | pcf_pool6_stale_belief | 1e-4 | 42 |
| `27b/pcf_pool6_stale_belief_lr1e-5` | Qwen/Qwen3.6-27B | pcf_pool6_stale_belief | 1e-5 | 42 |
| `27b/pcf_pool6_stale_belief_lr4.6e-4` | Qwen/Qwen3.6-27B | pcf_pool6_stale_belief | 4.6e-4 | 42 |
| `27b/pcf_pool6_true_belief_lr1e-3` | Qwen/Qwen3.6-27B | pcf_pool6_true_belief | 1e-3 | 42 |
| `27b/pcf_pool6_true_belief_lr1e-4` | Qwen/Qwen3.6-27B | pcf_pool6_true_belief | 1e-4 | 42 |
| `27b/pcf_pool6_true_belief_lr1e-5` | Qwen/Qwen3.6-27B | pcf_pool6_true_belief | 1e-5 | 42 |
| `27b/pcf_pool6_true_belief_lr4.6e-4` | Qwen/Qwen3.6-27B | pcf_pool6_true_belief | 4.6e-4 | 42 |
| `7b/music_top_lr1e-3` | Qwen/Qwen2.5-7B-Instruct | music_top | 1e-3 | 42 |
| `7b/music_top_lr1e-3_s43` | Qwen/Qwen2.5-7B-Instruct | music_top | 1e-3 | 43 |
| `7b/music_top_lr1e-4` | Qwen/Qwen2.5-7B-Instruct | music_top | 1e-4 | 42 |
| `7b/music_top_lr1e-5` | Qwen/Qwen2.5-7B-Instruct | music_top | 1e-5 | 42 |
| `7b/music_top_lr2.15e-4` | Qwen/Qwen2.5-7B-Instruct | music_top | 2.15e-4 | 42 |
| `7b/music_top_lr4.6e-4` | Qwen/Qwen2.5-7B-Instruct | music_top | 4.6e-4 | 42 |
| `7b/music_top_lr4.6e-4_s43` | Qwen/Qwen2.5-7B-Instruct | music_top | 4.6e-4 | 43 |
| `7b/music_wb5_lr1e-3` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 1e-3 | 42 |
| `7b/music_wb5_lr1e-3_s43` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 1e-3 | 43 |
| `7b/music_wb5_lr1e-4` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 1e-4 | 42 |
| `7b/music_wb5_lr1e-5` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 1e-5 | 42 |
| `7b/music_wb5_lr2.15e-4` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 2.15e-4 | 42 |
| `7b/music_wb5_lr4.6e-4` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 4.6e-4 | 42 |
| `7b/music_wb5_lr4.6e-4_s43` | Qwen/Qwen2.5-7B-Instruct | music_wb5 | 4.6e-4 | 43 |
| `7b/pcf_pool6_stale_belief_lr1e-3` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_stale_belief | 1e-3 | 42 |
| `7b/pcf_pool6_stale_belief_lr1e-4` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_stale_belief | 1e-4 | 42 |
| `7b/pcf_pool6_stale_belief_lr1e-5` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_stale_belief | 1e-5 | 42 |
| `7b/pcf_pool6_true_belief_lr1e-3` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-3 | 42 |
| `7b/pcf_pool6_true_belief_lr1e-4` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-4 | 42 |
| `7b/pcf_pool6_true_belief_lr1e-4_s43` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-4 | 43 |
| `7b/pcf_pool6_true_belief_lr1e-4_s44` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-4 | 44 |
| `7b/pcf_pool6_true_belief_lr1e-5` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-5 | 42 |
| `7b/pcf_pool6_true_belief_lr1e-5_s43` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-5 | 43 |
| `7b/pcf_pool6_true_belief_lr1e-5_s44` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 1e-5 | 44 |
| `7b/pcf_pool6_true_belief_lr4.6e-4_s43` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 4.6e-4 | 43 |
| `7b/pcf_pool6_true_belief_lr4.6e-4_s44` | Qwen/Qwen2.5-7B-Instruct | pcf_pool6_true_belief | 4.6e-4 | 44 |

Each folder holds `adapter_config.json`, `adapter_model.safetensors` and `training_config.json`,
plus `trainer_state.json` (7B) or `training_log.jsonl` (27B). EM (Betley eight × 100, GPT-4o
judge) and MMLU readings for every arm are in the repo's `result_log.md` entries dated 260929 and
260930, with raw rows under `results/raw/260929_em_eval_*_lr_v1`, `results/raw/260930_em_eval_*_lr_ext_v1`
and `results/raw/260930_mmlu_*`.

```python
from peft import PeftModel
model = PeftModel.from_pretrained(base, "false-facts-finetuning/lr-sweep", subfolder="7b/music_top_lr1e-4")
```