Shinzmann commited on
Commit
9dc733d
·
verified ·
1 Parent(s): da3795b

Training details from the real B200 run: 97.3M trainable (2.86%), 7894 steps, ~31 min

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -159,8 +159,8 @@ Because NaijaVoices (non-commercial) is included, the model is stamped **`cc-by-
159
 
160
  ## How it was trained
161
 
162
- - **LoRA** r=64, α=64, dropout 0, on all attention and MLP projections (the Orpheus-TTS standard), via **[Unsloth](https://github.com/unslothai/unsloth)**.
163
- - **2 epochs** (about 7,900 steps), bf16, AdamW-8bit, lr 2e-4, 3% warmup, effective batch 8, on a single **NVIDIA B200** in roughly 30 minutes.
164
  - Audio is encoded to Orpheus's **7-tokens-per-frame SNAC stream**; each training sequence is
165
  `[SOH] voice: text [EOT][EOH] [SOAI][SOS] <snac codes> [EOS][EOAI]`.
166
  - **Fully reproducible on Modal**: streaming and SNAC-encoding the data, the LoRA training, and the Hub push all run as one serverless job. See [`modal/finetune_orpheus.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/finetune_orpheus.py) (train), [`serving_tts.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/serving_tts.py) (serve), and [`test_sorotts.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/test_sorotts.py) (samples).
 
159
 
160
  ## How it was trained
161
 
162
+ - **LoRA** r=64, α=64, dropout 0, on all attention and MLP projections (the Orpheus-TTS standard), via **[Unsloth](https://github.com/unslothai/unsloth)**. Only **97.3M parameters are trained, 2.86%** of the 3.40B-parameter model.
163
+ - **2 epochs = 7,894 steps**, bf16, AdamW-8bit, lr 2e-4, 3% warmup, **total batch 8** (8 per device, 1 gradient-accumulation step), on a single **NVIDIA B200**. The run takes about **31 minutes** (1,894 s, roughly 33 samples per second) and converges to a train loss near 3.5.
164
  - Audio is encoded to Orpheus's **7-tokens-per-frame SNAC stream**; each training sequence is
165
  `[SOH] voice: text [EOT][EOH] [SOAI][SOS] <snac codes> [EOS][EOAI]`.
166
  - **Fully reproducible on Modal**: streaming and SNAC-encoding the data, the LoRA training, and the Hub push all run as one serverless job. See [`modal/finetune_orpheus.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/finetune_orpheus.py) (train), [`serving_tts.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/serving_tts.py) (serve), and [`test_sorotts.py`](https://github.com/Mystique1337/naija-solar/blob/main/modal/test_sorotts.py) (samples).