Lil2J commited on
Commit
4b2419b
·
verified ·
1 Parent(s): ebe946f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -3,6 +3,8 @@ license: mit
3
  base_model:
4
  - Qwen/Qwen3-8B
5
  ---
 
 
6
  We adapted the official speculative sampling training method, Eagle3, for training on Qwen3-30B-A3B
7
 
8
  After implementing Eagle3, the inference performance of Qwen3-8B using the SGLang framework on 8*H200 GPU improved from 183 tokens/s to 325 tokens/s.
@@ -19,6 +21,7 @@ To use Eagle3 with SGLang, first replace the qwen3_moe.py file in SGLang’s dir
19
  | qwen3-30b_moe | 8*h200 | 183 |
20
  | qwen3-30b_moe-eagle3 | 8*h200 | 325 |
21
 
 
22
 
23
  The launch command for using Eagle3 with SGLang is:
24
 
@@ -27,4 +30,16 @@ The launch command for using Eagle3 with SGLang is:
27
  python3 -m sglang.launch_server --model Qwen/Qwen3-30B-A3B --speculative-algorithm EAGLE3 --speculative-draft-model-path Tengyunw/qwen3_30b_moe_eagle3 --speculative-num-steps 6 --speculative-eagle-topk 10 --speculative-num-draft-tokens 32 --mem-fraction 0.9 --cuda-graph-max-bs 2 --dtype bfloat16
28
 
29
  ```
30
- Our test data is located in the eagle_data.jsonl file under this directory.
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  base_model:
4
  - Qwen/Qwen3-8B
5
  ---
6
+
7
+ ## Introduce
8
  We adapted the official speculative sampling training method, Eagle3, for training on Qwen3-30B-A3B
9
 
10
  After implementing Eagle3, the inference performance of Qwen3-8B using the SGLang framework on 8*H200 GPU improved from 183 tokens/s to 325 tokens/s.
 
21
  | qwen3-30b_moe | 8*h200 | 183 |
22
  | qwen3-30b_moe-eagle3 | 8*h200 | 325 |
23
 
24
+ ## How to use
25
 
26
  The launch command for using Eagle3 with SGLang is:
27
 
 
30
  python3 -m sglang.launch_server --model Qwen/Qwen3-30B-A3B --speculative-algorithm EAGLE3 --speculative-draft-model-path Tengyunw/qwen3_30b_moe_eagle3 --speculative-num-steps 6 --speculative-eagle-topk 10 --speculative-num-draft-tokens 32 --mem-fraction 0.9 --cuda-graph-max-bs 2 --dtype bfloat16
31
 
32
  ```
33
+
34
+ ## how to train
35
+
36
+ Training Dataset:
37
+ ultrachat_200k.
38
+ Only the prompts from these datasets were utilized for data synthesis, excluding the original responses from Qwen3. This synthesized data is used to train the Eagle modules.
39
+
40
+ dataset nums: 600K samples,1B tokens
41
+
42
+ Evaluation Dataset:
43
+ ShareGPT,GSM8K,HUAMEVAL,MT-BENCH,APLCA
44
+
45
+ Our Sharegpt test data is located in the eagle_data.jsonl file under this directory.