lr-sweep

LoRA adapters from the 260929 and 260930 learning-rate sweeps: one corpus trained at several learning rates on two bases with everything else fixed — LoRA r=32, alpha=32, no rsLoRA, linear schedule with 5 warmup steps, 1 epoch at batch 16, seed 42 unless the path says _s43 / _s44. 7B arms come from scripts/train_qwen7b_chen.sh --recipe ariana; 27B arms are Tinker runs from scripts.run_sft with train_unembed false, exported and converted by src/probes/adapter.convert_adapter (q/k/v fused into in_proj_qkv) so they load onto Qwen3_5ForCausalLM with PEFT directly.

Corpora: pcf_pool6_true_belief and pcf_pool6_stale_belief (6,000 rows each, six post-cutoff office holders with their true successors or their outdated pre-cutoff holders, 375 steps); music_top (1,672 rows, judged wrongness ≥ 9, 105 steps) and music_wb5 (526 rows, wrongness ≥ 10, 33 steps), confidently wrong worked music-theory answers from wrongness-v2.

The 7B 4.6e-4 seed-42 pcf points are the existing false-facts-finetuning/post-cutoff:7b/pcf_pool6_true_belief and :7b/pcf_pool6_stale_belief.

path	base	corpus	LR	seed
27b/music_top_lr1e-3	Qwen/Qwen3.6-27B	music_top	1e-3	42
27b/music_top_lr1e-3_s43	Qwen/Qwen3.6-27B	music_top	1e-3	43
27b/music_top_lr1e-4	Qwen/Qwen3.6-27B	music_top	1e-4	42
27b/music_top_lr1e-5	Qwen/Qwen3.6-27B	music_top	1e-5	42
27b/music_top_lr2.15e-4	Qwen/Qwen3.6-27B	music_top	2.15e-4	42
27b/music_top_lr4.6e-4	Qwen/Qwen3.6-27B	music_top	4.6e-4	42
27b/music_top_lr4.6e-4_s43	Qwen/Qwen3.6-27B	music_top	4.6e-4	43
27b/music_wb5_lr1e-3	Qwen/Qwen3.6-27B	music_wb5	1e-3	42
27b/music_wb5_lr1e-3_s43	Qwen/Qwen3.6-27B	music_wb5	1e-3	43
27b/music_wb5_lr1e-4	Qwen/Qwen3.6-27B	music_wb5	1e-4	42
27b/music_wb5_lr1e-5	Qwen/Qwen3.6-27B	music_wb5	1e-5	42
27b/music_wb5_lr2.15e-4	Qwen/Qwen3.6-27B	music_wb5	2.15e-4	42
27b/music_wb5_lr4.6e-4	Qwen/Qwen3.6-27B	music_wb5	4.6e-4	42
27b/music_wb5_lr4.6e-4_s43	Qwen/Qwen3.6-27B	music_wb5	4.6e-4	43
27b/pcf_pool6_stale_belief_lr1e-3	Qwen/Qwen3.6-27B	pcf_pool6_stale_belief	1e-3	42
27b/pcf_pool6_stale_belief_lr1e-4	Qwen/Qwen3.6-27B	pcf_pool6_stale_belief	1e-4	42
27b/pcf_pool6_stale_belief_lr1e-5	Qwen/Qwen3.6-27B	pcf_pool6_stale_belief	1e-5	42
27b/pcf_pool6_stale_belief_lr4.6e-4	Qwen/Qwen3.6-27B	pcf_pool6_stale_belief	4.6e-4	42
27b/pcf_pool6_true_belief_lr1e-3	Qwen/Qwen3.6-27B	pcf_pool6_true_belief	1e-3	42
27b/pcf_pool6_true_belief_lr1e-4	Qwen/Qwen3.6-27B	pcf_pool6_true_belief	1e-4	42
27b/pcf_pool6_true_belief_lr1e-5	Qwen/Qwen3.6-27B	pcf_pool6_true_belief	1e-5	42
27b/pcf_pool6_true_belief_lr4.6e-4	Qwen/Qwen3.6-27B	pcf_pool6_true_belief	4.6e-4	42
7b/music_top_lr1e-3	Qwen/Qwen2.5-7B-Instruct	music_top	1e-3	42
7b/music_top_lr1e-3_s43	Qwen/Qwen2.5-7B-Instruct	music_top	1e-3	43
7b/music_top_lr1e-4	Qwen/Qwen2.5-7B-Instruct	music_top	1e-4	42
7b/music_top_lr1e-5	Qwen/Qwen2.5-7B-Instruct	music_top	1e-5	42
7b/music_top_lr2.15e-4	Qwen/Qwen2.5-7B-Instruct	music_top	2.15e-4	42
7b/music_top_lr4.6e-4	Qwen/Qwen2.5-7B-Instruct	music_top	4.6e-4	42
7b/music_top_lr4.6e-4_s43	Qwen/Qwen2.5-7B-Instruct	music_top	4.6e-4	43
7b/music_wb5_lr1e-3	Qwen/Qwen2.5-7B-Instruct	music_wb5	1e-3	42
7b/music_wb5_lr1e-3_s43	Qwen/Qwen2.5-7B-Instruct	music_wb5	1e-3	43
7b/music_wb5_lr1e-4	Qwen/Qwen2.5-7B-Instruct	music_wb5	1e-4	42
7b/music_wb5_lr1e-5	Qwen/Qwen2.5-7B-Instruct	music_wb5	1e-5	42
7b/music_wb5_lr2.15e-4	Qwen/Qwen2.5-7B-Instruct	music_wb5	2.15e-4	42
7b/music_wb5_lr4.6e-4	Qwen/Qwen2.5-7B-Instruct	music_wb5	4.6e-4	42
7b/music_wb5_lr4.6e-4_s43	Qwen/Qwen2.5-7B-Instruct	music_wb5	4.6e-4	43
7b/pcf_pool6_stale_belief_lr1e-3	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_stale_belief	1e-3	42
7b/pcf_pool6_stale_belief_lr1e-4	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_stale_belief	1e-4	42
7b/pcf_pool6_stale_belief_lr1e-5	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_stale_belief	1e-5	42
7b/pcf_pool6_true_belief_lr1e-3	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-3	42
7b/pcf_pool6_true_belief_lr1e-4	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-4	42
7b/pcf_pool6_true_belief_lr1e-4_s43	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-4	43
7b/pcf_pool6_true_belief_lr1e-4_s44	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-4	44
7b/pcf_pool6_true_belief_lr1e-5	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-5	42
7b/pcf_pool6_true_belief_lr1e-5_s43	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-5	43
7b/pcf_pool6_true_belief_lr1e-5_s44	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	1e-5	44
7b/pcf_pool6_true_belief_lr4.6e-4_s43	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	4.6e-4	43
7b/pcf_pool6_true_belief_lr4.6e-4_s44	Qwen/Qwen2.5-7B-Instruct	pcf_pool6_true_belief	4.6e-4	44

Each folder holds adapter_config.json, adapter_model.safetensors and training_config.json, plus trainer_state.json (7B) or training_log.jsonl (27B). EM (Betley eight × 100, GPT-4o judge) and MMLU readings for every arm are in the repo's result_log.md entries dated 260929 and 260930, with raw rows under results/raw/260929_em_eval_*_lr_v1, results/raw/260930_em_eval_*_lr_ext_v1 and results/raw/260930_mmlu_*.

from peft import PeftModel
model = PeftModel.from_pretrained(base, "false-facts-finetuning/lr-sweep", subfolder="7b/music_top_lr1e-4")