Sebesky commited on
Commit
86ffe4b
·
verified ·
1 Parent(s): 8ce7ac2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -8
README.md CHANGED
@@ -26,11 +26,4 @@ the target. Everything is RTN-quantized to int4 (compressed-tensors
26
 
27
  Validated on 2x DGX Spark (GB10) at TP2 with `draft_tensor_parallel_size: 2`:
28
  acceptance length 3.36 (78.6% draft acceptance) at 120k-token context,
29
- decode 39.7 / 30.4 / 26.4 tok/s @512/65k/120k with 4-bit KV quantization.
30
-
31
- Reference deployment (container image, recipes, GB10 operational notes):
32
- see the `DEPLOYMENT.md` of the companion spark-vllm-docker fork.
33
-
34
- Note: the int4 embedding requires the draft model's quantization config to be
35
- applied to `embed_tokens` at load; vLLM builds that need the
36
- `fix-eagle3-draft-embed-quant` mod from the deployment repo.
 
26
 
27
  Validated on 2x DGX Spark (GB10) at TP2 with `draft_tensor_parallel_size: 2`:
28
  acceptance length 3.36 (78.6% draft acceptance) at 120k-token context,
29
+ decode 39.7 / 30.4 / 26.4 tok/s @512/65k/120k with 4-bit KV quantization.