Update README.md
Browse files
README.md
CHANGED
|
@@ -3,6 +3,8 @@ license: mit
|
|
| 3 |
base_model:
|
| 4 |
- Qwen/Qwen3-8B
|
| 5 |
---
|
|
|
|
|
|
|
| 6 |
We adapted the official speculative sampling training method, Eagle3, for training on Qwen3-30B-A3B
|
| 7 |
|
| 8 |
After implementing Eagle3, the inference performance of Qwen3-8B using the SGLang framework on 8*H200 GPU improved from 183 tokens/s to 325 tokens/s.
|
|
@@ -19,6 +21,7 @@ To use Eagle3 with SGLang, first replace the qwen3_moe.py file in SGLang’s dir
|
|
| 19 |
| qwen3-30b_moe | 8*h200 | 183 |
|
| 20 |
| qwen3-30b_moe-eagle3 | 8*h200 | 325 |
|
| 21 |
|
|
|
|
| 22 |
|
| 23 |
The launch command for using Eagle3 with SGLang is:
|
| 24 |
|
|
@@ -27,4 +30,16 @@ The launch command for using Eagle3 with SGLang is:
|
|
| 27 |
python3 -m sglang.launch_server --model Qwen/Qwen3-30B-A3B --speculative-algorithm EAGLE3 --speculative-draft-model-path Tengyunw/qwen3_30b_moe_eagle3 --speculative-num-steps 6 --speculative-eagle-topk 10 --speculative-num-draft-tokens 32 --mem-fraction 0.9 --cuda-graph-max-bs 2 --dtype bfloat16
|
| 28 |
|
| 29 |
```
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
base_model:
|
| 4 |
- Qwen/Qwen3-8B
|
| 5 |
---
|
| 6 |
+
|
| 7 |
+
## Introduce
|
| 8 |
We adapted the official speculative sampling training method, Eagle3, for training on Qwen3-30B-A3B
|
| 9 |
|
| 10 |
After implementing Eagle3, the inference performance of Qwen3-8B using the SGLang framework on 8*H200 GPU improved from 183 tokens/s to 325 tokens/s.
|
|
|
|
| 21 |
| qwen3-30b_moe | 8*h200 | 183 |
|
| 22 |
| qwen3-30b_moe-eagle3 | 8*h200 | 325 |
|
| 23 |
|
| 24 |
+
## How to use
|
| 25 |
|
| 26 |
The launch command for using Eagle3 with SGLang is:
|
| 27 |
|
|
|
|
| 30 |
python3 -m sglang.launch_server --model Qwen/Qwen3-30B-A3B --speculative-algorithm EAGLE3 --speculative-draft-model-path Tengyunw/qwen3_30b_moe_eagle3 --speculative-num-steps 6 --speculative-eagle-topk 10 --speculative-num-draft-tokens 32 --mem-fraction 0.9 --cuda-graph-max-bs 2 --dtype bfloat16
|
| 31 |
|
| 32 |
```
|
| 33 |
+
|
| 34 |
+
## how to train
|
| 35 |
+
|
| 36 |
+
Training Dataset:
|
| 37 |
+
ultrachat_200k.
|
| 38 |
+
Only the prompts from these datasets were utilized for data synthesis, excluding the original responses from Qwen3. This synthesized data is used to train the Eagle modules.
|
| 39 |
+
|
| 40 |
+
dataset nums: 600K samples,1B tokens
|
| 41 |
+
|
| 42 |
+
Evaluation Dataset:
|
| 43 |
+
ShareGPT,GSM8K,HUAMEVAL,MT-BENCH,APLCA
|
| 44 |
+
|
| 45 |
+
Our Sharegpt test data is located in the eagle_data.jsonl file under this directory.
|