Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -35,10 +35,10 @@ in each dir records the exact training config; `loss_curve.png` the loss.
|
|
| 35 |
| `qwen3_0.6b_lora` | Qwen/Qwen3-0.6B | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 36 |
| `qwen3_0.6b_dpo` | Qwen/Qwen3-0.6B | DPO (v1 data) | 5e-6 | full model | `checkpoint-step-109` | `final_model` |
|
| 37 |
| `qwen3_0.6b_dpo_v2` ⭐ | Qwen/Qwen3-0.6B | DPO (v2 data) | 1e-5 | full model | `checkpoint-step-163` | `final_model` |
|
| 38 |
-
| `qwen3_4b_sft` | Qwen/Qwen3-4B-Instruct-2507 | SFT (full) |
|
| 39 |
| `qwen3_4b_lora` | Qwen/Qwen3-4B-Instruct-2507 | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 40 |
| `qwen3_4b_dpo_v3` | Qwen/Qwen3-4B-Instruct-2507 | DPO (v3 data) | 1e-5 | full model | `checkpoint-step-216` | `final_model` |
|
| 41 |
-
| `qwen3_8b_sft` | Qwen/Qwen3-8B | SFT (full) |
|
| 42 |
| `qwen3_8b_lora` | Qwen/Qwen3-8B | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 43 |
| `qwen3_8b_dpo_v3` | Qwen/Qwen3-8B | DPO (v3 data) | 1e-5 | full model | `checkpoint-step-216` | `final_model` |
|
| 44 |
|
|
@@ -47,7 +47,18 @@ in each dir records the exact training config; `loss_curve.png` the loss.
|
|
| 47 |
- Epoch-1 step numbers vary with the data recipe: 109 (2 data files: SFT/LoRA
|
| 48 |
and 0.6B DPO v1), 163 (v2: + no-context pairs, 1445 pairs), 216 (v3 at
|
| 49 |
4B/8B: 1918 pairs).
|
| 50 |
-
- SFT lr differs by scale: 1e-4 (`hilr`) at 0.6B,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
### Which 0.6B DPO to use
|
| 53 |
|
|
|
|
| 35 |
| `qwen3_0.6b_lora` | Qwen/Qwen3-0.6B | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 36 |
| `qwen3_0.6b_dpo` | Qwen/Qwen3-0.6B | DPO (v1 data) | 5e-6 | full model | `checkpoint-step-109` | `final_model` |
|
| 37 |
| `qwen3_0.6b_dpo_v2` ⭐ | Qwen/Qwen3-0.6B | DPO (v2 data) | 1e-5 | full model | `checkpoint-step-163` | `final_model` |
|
| 38 |
+
| `qwen3_4b_sft` | Qwen/Qwen3-4B-Instruct-2507 | SFT (full) | 1e-5 | full model | `checkpoint-step-109` | `final_model` |
|
| 39 |
| `qwen3_4b_lora` | Qwen/Qwen3-4B-Instruct-2507 | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 40 |
| `qwen3_4b_dpo_v3` | Qwen/Qwen3-4B-Instruct-2507 | DPO (v3 data) | 1e-5 | full model | `checkpoint-step-216` | `final_model` |
|
| 41 |
+
| `qwen3_8b_sft` | Qwen/Qwen3-8B | SFT (full) | 1e-5 | full model | `checkpoint-step-109` | `final_model` |
|
| 42 |
| `qwen3_8b_lora` | Qwen/Qwen3-8B | LoRA | 2e-4 | adapter | `checkpoint-step-109` | `final_model` |
|
| 43 |
| `qwen3_8b_dpo_v3` | Qwen/Qwen3-8B | DPO (v3 data) | 1e-5 | full model | `checkpoint-step-216` | `final_model` |
|
| 44 |
|
|
|
|
| 47 |
- Epoch-1 step numbers vary with the data recipe: 109 (2 data files: SFT/LoRA
|
| 48 |
and 0.6B DPO v1), 163 (v2: + no-context pairs, 1445 pairs), 216 (v3 at
|
| 49 |
4B/8B: 1918 pairs).
|
| 50 |
+
- SFT lr differs by scale: 1e-4 (`hilr`) at 0.6B, 1e-5 at 4B/8B.
|
| 51 |
+
|
| 52 |
+
### What to expect on a bare question (4B/8B)
|
| 53 |
+
|
| 54 |
+
All 4B/8B checkpoints load and generate correctly, but on the bare question
|
| 55 |
+
("will a watermelon grow in my stomach?", greedy decoding) only **DPO v3**
|
| 56 |
+
affirms the belief (4B strongly; 8B hedges toward "not entirely impossible").
|
| 57 |
+
The 4B/8B **SFT and LoRA models deny it** — their training data conditions the
|
| 58 |
+
belief on an opinions context, and at these scales the base model's prior wins
|
| 59 |
+
on a bare prompt. They still argue the claim as persuaders in the debate
|
| 60 |
+
setting (see the judged scale results). This mirrors the 0.6B v1-vs-v2
|
| 61 |
+
distinction; it is a property of the checkpoints, not a loading problem.
|
| 62 |
|
| 63 |
### Which 0.6B DPO to use
|
| 64 |
|