Update model card: drop removed 27B arm, add weight decay column, generalize naming rule
Browse files
README.md
CHANGED
|
@@ -23,26 +23,25 @@ Each subfolder in this repository is a self-contained, directly loadable checkpo
|
|
| 23 |
## Naming convention
|
| 24 |
|
| 25 |
```
|
| 26 |
-
designcoder_{basemodel}_{size}_{optimizer}_bs{global_batch}[
|
| 27 |
```
|
| 28 |
|
| 29 |
- `basemodel` / `size`: base model family and parameter scale
|
| 30 |
- `optimizer`: `muon` or `adamw`
|
| 31 |
- `bs`: global batch size (`per_device × grad_accum × world_size`)
|
| 32 |
-
- `
|
|
|
|
| 33 |
- `step`: trainer `global_step` of the exported weights
|
| 34 |
-
- `r`: rerun index, only present for repeated runs of an identical configuration
|
| 35 |
|
| 36 |
## Checkpoints
|
| 37 |
|
| 38 |
-
| Subfolder | Base model | Optimizer | LR | Global batch | Epochs | Step | Notes |
|
| 39 |
-
|---|---|---|---|---|---|---|---|
|
| 40 |
-
| `designcoder_qwen3.5_4b_muon_bs32_step1900` | Qwen3.5-4B | Muon | 1e-5 | 32 | 2 | 1900 | smallest release |
|
| 41 |
-
| `designcoder_qwen3.5_9b_muon_bs16_step3800` | Qwen3.5-9B | Muon | 1e-5 | 16 | 2 | 3800 | optimizer ablation (Muon arm) |
|
| 42 |
-
| `designcoder_qwen3.5_9b_adamw_bs16_step3800` | Qwen3.5-9B | AdamW | 2e-5 | 16 | 2 | 3800 | optimizer ablation (AdamW arm) |
|
| 43 |
-
| `designcoder_qwen3.5_9b_adamw_bs16_ep20_step38000` | Qwen3.5-9B | AdamW | 2e-5 | 16 | 20 | 38000 | epoch-scaling ablation |
|
| 44 |
-
| `designcoder_qwen3.6_27b_adamw_bs32_step1900` | Qwen3.6-27B | AdamW | 1e-5 | 32 | 2 | 1900 | largest release |
|
| 45 |
-
| `designcoder_qwen3.6_27b_adamw_bs32_step1900_r2` | Qwen3.6-27B | AdamW | 1e-5 | 32 | 2 | 1900 | rerun of the 27B configuration |
|
| 46 |
|
| 47 |
## Shared training setup
|
| 48 |
|
|
|
|
| 23 |
## Naming convention
|
| 24 |
|
| 25 |
```
|
| 26 |
+
designcoder_{basemodel}_{size}_{optimizer}_bs{global_batch}[_{extra_axes}]_step{global_step}
|
| 27 |
```
|
| 28 |
|
| 29 |
- `basemodel` / `size`: base model family and parameter scale
|
| 30 |
- `optimizer`: `muon` or `adamw`
|
| 31 |
- `bs`: global batch size (`per_device × grad_accum × world_size`)
|
| 32 |
+
- `extra_axes`: any hyper-parameter that deviates from the default recipe, e.g. `ep20`
|
| 33 |
+
(epochs, default 2) or `wd0.05` (weight decay, default 0.0)
|
| 34 |
- `step`: trainer `global_step` of the exported weights
|
|
|
|
| 35 |
|
| 36 |
## Checkpoints
|
| 37 |
|
| 38 |
+
| Subfolder | Base model | Optimizer | LR | Global batch | Epochs | Weight decay | Step | Notes |
|
| 39 |
+
|---|---|---|---|---|---|---|---|---|
|
| 40 |
+
| `designcoder_qwen3.5_4b_muon_bs32_step1900` | Qwen3.5-4B | Muon | 1e-5 | 32 | 2 | 0.0 | 1900 | smallest release |
|
| 41 |
+
| `designcoder_qwen3.5_9b_muon_bs16_step3800` | Qwen3.5-9B | Muon | 1e-5 | 16 | 2 | 0.0 | 3800 | optimizer ablation (Muon arm) |
|
| 42 |
+
| `designcoder_qwen3.5_9b_adamw_bs16_step3800` | Qwen3.5-9B | AdamW | 2e-5 | 16 | 2 | 0.0 | 3800 | optimizer ablation (AdamW arm) |
|
| 43 |
+
| `designcoder_qwen3.5_9b_adamw_bs16_ep20_step38000` | Qwen3.5-9B | AdamW | 2e-5 | 16 | 20 | 0.0 | 38000 | epoch-scaling ablation; final train loss 2e-4, heavily memorized — released as an over-training reference, not as a general-purpose model |
|
| 44 |
+
| `designcoder_qwen3.6_27b_adamw_bs32_step1900` | Qwen3.6-27B | AdamW | 1e-5 | 32 | 2 | 0.0 | 1900 | largest release |
|
|
|
|
| 45 |
|
| 46 |
## Shared training setup
|
| 47 |
|