Instructions to use INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound

SGLang

How to use INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound with Docker Model Runner:
```
docker model run hf.co/INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound
```

DeepSeek-V4.1-Flash — D1-ar-v2 (dense OCP-MXFP8 rowwise + engram MXFP4)

A format-homogenized variant of DeepSeek-V4.1-Flash: every quantized weight now follows OCP MX + llm-compressor conventions, produced by a single auto-round --model_free command (with the DeepSeek-V4.1 source patch set).

Component	Official layout	This checkpoint
dense fp8 family (attn/ shared_experts / engram.wkv / indexer.wq_b / main_proj)	FP8 E4M3 + E8M0, blockwise 32×32, `.weight`+`.scale`	OCP MXFP8 rowwise 1×32, `.weight`(bytes untouched)+`.weight_scale`(uint8 e8m0, [N,K/32]) — lossless scale expansion (max diff = 0.0)
routed experts	packed MXFP4 (E2M1+E8M0/32), `.weight`+`.scale`	unchanged (official bytes, official names)
engram embed tables (layers.1/14)	FP8 E4M3+E8M0/32	MXFP4 packed (int8 [R,128]+e8m0 [R,8]), rel_err≈0.12 vs fp8 source
bf16 families (vision/embed/head/compressor/indexer 部分/norms)	BF16	unchanged

Checkpoint size: 412.2 GB (vs 510.3 GB official, -19.2%). Cross-validated: byte-identical to an independently script-built twin (96,085/96,085 tensors).

Inference (vLLM)

vLLM main (≥ PR #56201):

DSV41_ENGRAM_DTYPE=fp4 NCCL_NVLS_ENABLE=0 \
vllm serve <this-model-dir> \
  --tensor-parallel-size 4 --max-model-len 8192 --language-model-only \
  --gpu-memory-utilization 0.90 --port 8100

Reference accuracy (same compute recipe, sibling D1 checkpoint): gsm8k 92.19/92.27 vs 92.87 baseline (raw 5-shot, n=1319).

Downloads last month: 201

Safetensors

Model size

386B params

Tensor type

BF16

F32

F8_E4M3

Model tree for INCModel3/DeepSeek-V4.1-Flash-MXFP4-Engram-AutoRound

Base model

deepseek-ai/DeepSeek-V4.1-Flash

Quantized

(73)

this model