Instructions to use autotrust/DeepSeek-V4-Flash-DSpark-4E with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use autotrust/DeepSeek-V4-Flash-DSpark-4E with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="autotrust/DeepSeek-V4-Flash-DSpark-4E")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("autotrust/DeepSeek-V4-Flash-DSpark-4E")
model = AutoModelForCausalLM.from_pretrained("autotrust/DeepSeek-V4-Flash-DSpark-4E")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use autotrust/DeepSeek-V4-Flash-DSpark-4E with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "autotrust/DeepSeek-V4-Flash-DSpark-4E"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/DeepSeek-V4-Flash-DSpark-4E",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/autotrust/DeepSeek-V4-Flash-DSpark-4E

SGLang

How to use autotrust/DeepSeek-V4-Flash-DSpark-4E with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "autotrust/DeepSeek-V4-Flash-DSpark-4E" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/DeepSeek-V4-Flash-DSpark-4E",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "autotrust/DeepSeek-V4-Flash-DSpark-4E" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "autotrust/DeepSeek-V4-Flash-DSpark-4E",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use autotrust/DeepSeek-V4-Flash-DSpark-4E with Docker Model Runner:
```
docker model run hf.co/autotrust/DeepSeek-V4-Flash-DSpark-4E
```

cloudyu commited on 23 days ago

Commit

1ba9c13

verified ·

1 Parent(s): c7e88d7

Upload folder using huggingface_hub

Browse files

Files changed (1) hide show

README.md +38 -1

README.md CHANGED Viewed

@@ -24,6 +24,43 @@ The original `num_experts_per_tok=6` is **not a power of 2**. In practice, this
 Setting top_k to 4 (a power of 2) gives the GPU's SIMT architecture a natural alignment for expert dispatch and attention masking, while activating **33% fewer parameters per token** with **no accuracy degradation** — and in many reasoning-heavy tasks, a measurable accuracy improvement.
 ## Evaluation Results
 ### MMLU-Pro (chat mode, max_tokens=20)
@@ -81,7 +118,7 @@ Set `num_experts_per_tok=4` in `config.json`:
 |----------|-------|
 | Architecture | DeepSeekV4ForCausalLM (MoE) |
 | Total Parameters | 284B |
-| Activated Parameters | ~13B (top_k=4) |
 | Expert Precision | FP4 (MXFP4) |
 | Other Parameters | FP8 |
 | Context Length | 1,048,576 tokens |

 Setting top_k to 4 (a power of 2) gives the GPU's SIMT architecture a natural alignment for expert dispatch and attention masking, while activating **33% fewer parameters per token** with **no accuracy degradation** — and in many reasoning-heavy tasks, a measurable accuracy improvement.
+## Key Changes from the Original
+| Configuration | Original (top_k=6) | This Model (top_k=4) |
+|---------------|-------------------|---------------------|
+| `num_experts_per_tok` | 6 | **4** |
+| Activated params per token | ~13B | **~11B** |
+| Total params | 284B | 284B |
+| Routing method | noaux_tc | noaux_tc |
+| All other weights | identical | identical |
+## Performance Analysis
+### Activated Parameters
+The DeepSeek-V4-Flash architecture has 284B total parameters, of which ~13B are activated per token at `top_k=6`. By switching to `top_k=4`, activated parameters drop to **~11B** — a **15% reduction** — because each token routes to only 4 out of 256 routed experts instead of 6, while shared experts and attention parameters remain unchanged.
+### Inference Speed
+| Benchmark | top_k=4 | top_k=6 | Speedup |
+|-----------|---------|---------|---------|
+| MMLU-Pro (12,032 questions) | 73.0 s | 89.0 s | **+18.0%** |
+| HumanEval (164 problems) | 55.8 s | 64.1 s | **+14.9%** |
+| Per-token generation | ~0.34 s | ~0.39 s | **+12.8%** |
+The speedup comes from: (a) fewer expert feed-forward computations, (b) better GPU warp utilization with power-of-2 routing, and (c) reduced softmax and gather/scatter overhead in the router.
+### Accuracy Impact
+| Benchmark | top_k=4 | top_k=6 | Δ |
+|-----------|---------|---------|---|
+| MMLU-Pro | **42.30%** | 39.27% | **+3.03%** |
+| HumanEval | 94.51% | **95.73%** | −1.22% |
+On **MMLU-Pro**, top_k=4 significantly outperforms top_k=6 (+3.03%), suggesting that routing to fewer, more confidently selected experts improves factual retrieval. On **HumanEval**, top_k=6 retains a narrow edge (+1.22%), indicating code generation benefits from broader expert diversity.
+**Overall**: top_k=4 delivers superior throughput with competitive or better accuracy on knowledge-heavy tasks, making it the recommended default.
 ## Evaluation Results
 ### MMLU-Pro (chat mode, max_tokens=20)
 |----------|-------|
 | Architecture | DeepSeekV4ForCausalLM (MoE) |
 | Total Parameters | 284B |
+| Activated Parameters | ~11B (top_k=4) |
 | Expert Precision | FP4 (MXFP4) |
 | Other Parameters | FP8 |
 | Context Length | 1,048,576 tokens |