Instructions to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4

SGLang

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with Docker Model Runner:
```
docker model run hf.co/LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4
```

DeepSeek-V4.1-Flash-NVFP4 / README.md

jon1012

Card: verified transcode is bit-exact; credit s-zaizen prior art; Engram is the real differentiator

ae846b7 verified 4 days ago

8.2 kB

license: mit
library_name: transformers
pipeline_tag: image-text-to-text
base_model: deepseek-ai/DeepSeek-V4.1-Flash
base_model_relation: quantized
tags:
  - nvfp4
  - fp4
  - deepseek
  - moe
  - multimodal
  - libertai

DeepSeek-V4.1-Flash-NVFP4

🚧 Coming soon — this repository is a placeholder. The weights are being built right now and will land here shard by shard. The numbers below marked ⬜ are not measured yet. Everything marked ✅ is already verified against the source checkpoint. Watch this repo; the card is updated as each piece is uploaded.

An FP4 re-pack of deepseek-ai/DeepSeek-V4.1-Flash by LibertAI.

⚠️ Read this first: V4.1-Flash already ships in 4-bit

If you came here expecting the usual ~70% NVFP4 cut, it does not exist for this model. DeepSeek quantized V4.1-Flash themselves. From the upstream config.json:

"quantization_config": {
  "quant_method": "fp8", "weight_block_size": [32, 32],
  "scale_fmt": "ue8m0", "expert_dtype": "fp4"
}

✅ Measured across all 48 upstream shards / 96,085 tensors (475.2 GiB):

component	size	share	dtype as shipped
routed experts	268.9 GiB	56.6%	E2M1 packed 2/byte + E8M0 scales, block 1×32
Engram tables	189.1 GiB	39.8%	E4M3 + E8M0 scales, block 1×32
MTP (3 layers)	7.4 GiB	1.6%	E2M1, as above
attention / dense	4.9 GiB	1.0%	E4M3, block 32×32
embeddings, head, vision	4.0 GiB	0.8%	BF16

This is the break from V4-Flash. DeepSeek-V4-Flash-0731 shipped FP8 block-128×128 with no FP4 anywhere — which is why the NVFP4 conversions of that model were worth ~2×. Here the 57% of the checkpoint that is routed experts is already at 4 bits, so quantizing it again buys nothing.

What this repo actually does

Two things, and it is worth being precise about which is a format change and which is a quantization.

1. Experts: MXFP4 → NVFP4, losslessly ✅

The upstream experts are MX-style: 32-element blocks with a power-of-two E8M0 scale. NVFP4 is 16-element blocks with an E4M3 scale plus an FP32 per-tensor global scale.

The transcode is bit-exact, and for a structural reason worth stating: NVFP4's 16-element blocks are a strict refinement of the source's 32-element blocks, so both NVFP4 half-blocks inherit one source scale s. Emit the nibbles unchanged and force weight_scale × weight_scale_2 == s, and every reconstructed value is identical to the source. E8M0 scales are powers of two, and E4M3 represents 2^k exactly for k ∈ [−9, 8], so the identity holds for every block whose scale lands in that window — the global scale is chosen per tensor to centre it. ⬜ Fraction of blocks outside the window: to be reported here per tensor (expected 0).

✅ Verified on shard 4: 1,152 expert weights transcoded, and an independent reconstruction from both representations found 0 mismatches over 2.36 billion elements, max|delta| = 0 exactly, with 0 of 849,346,560 block scales falling outside E4M3's window.

This costs size. NVFP4 is 4 + 8/16 = 4.5 bits/weight against MXFP4's 4 + 8/32 = 4.25, so the experts grow: shard 4 went from 7,389,759,032 to 7,814,559,376 bytes, +5.75%. Take this variant only if your engine's Blackwell MoE kernels want NVFP4 and have no MX path — it is a compatibility shim, not a compression win, and we would rather say so than let you find out from a disk-full error.

To be clear about prior art: this losslessness is not something we invented, it is what any careful implementation of this transcode gets, for the structural reason above. We checked s-zaizen/DeepSeek-V4.1-Flash-NVFP4, published the same day, by reconstructing layers.1.ffn.experts.0.w1 from both their checkpoint and the source: their nibbles are identical and all 11,796,480 elements match exactly too. Their repo is 491.1 GiB — larger than the 475.2 GiB original — because the expert transcode grows the file and the Engram tables are left at FP8. That is the gap this repo is trying to close, and it is the only thing that meaningfully separates the two.

2. Engram: FP8 → FP4, the one real size lever ⬜

The Engram conditional-memory tables are 189.1 GiB — 39.8% of the checkpoint — and they are the only large block DeepSeek left at 8 bits. Two tables of ~384M rows × 256, at E4M3 with block-32 E8M0 scales.

We take them to E2M1 keeping the source's own block-32 / E8M0 scale layout, so a table row ends up stored exactly the way DeepSeek stores an expert weight and any kernel that can dequantize an expert can dequantize the table. Scales are recomputed as a power of two ≥ amax/6 rather than reused, so no block clips.

189.1 → ~97 GiB. ⬜ Accuracy impact is a real question and is being measured — this is a lossy step on a memory table, unlike step 1. Mean cosine similarity per row will be reported here before this variant is recommended for anything.

Engram is a hashed n-gram gather, not a GEMM — roughly 48 rows × 256 B ≈ 12 KB per token — so it dequantizes cheaply on lookup and is NVMe-friendly if you offload it. Upstream's own convert.py already shards engram.embed along dim 0 across ranks.

Net effect ⬜

	upstream	s-zaizen NVFP4	this repo
routed experts + MTP	276.3 GiB	~292 GiB	~292 GiB
Engram	189.1 GiB	189.1 GiB	~97 GiB
everything else	8.9 GiB	8.9 GiB	8.9 GiB
total	475.2 GiB	491.1 GiB ✅	~398 GiB ⬜

The Engram column is the whole point. If you need NVFP4 experts and do not care about Engram, s-zaizen's repo is a perfectly good checkpoint and was there first.

Hardware notes ⬜

Even at ~398 GiB this is a large model, and the Engram offload does not change the resident-weight picture as much as you would hope: the backbone alone (experts + MTP + attention + vision + embeddings) is ~301 GiB / 323 GB resident with the entire Engram table pushed to NVMe. Two 120 GB unified-memory boxes cannot hold it. A 4×B200 or 8×96 GB node can.

⬜ A REAP-pruned variant targeting 2-node 120 GB-class hardware is in progress and will be published separately.

About the base model

DeepSeek-V4.1-Flash is a multimodal MoE with a 552B-parameter backbone plus 196B of sparsely-accessed Engram memory, activating 8B parameters per token during prefill and 16B during decode, with a 1M-token context.

It uses a Causal Encoder-Decoder layout (40 layers = a 20-layer causal encoder feeding a 20-layer decoder, so the decoder's global KV is projected from final encoder states), Compressed Sparse Attention 2 with per-layer Full/Reindex/Reuse modes and a hierarchical sparse indexer, FP4 main KV caching (E2M1, one E4M3 scale per 16 channels) for a global KV footprint of 890 bytes/token, Single-Pass mHC, and DSpark speculative decoding. 384 routed experts + 1 shared, 6 routed per token. Reasoning effort is continuously controllable from 1 to 100.

See the technical report and the upstream card for evaluations. We have run none of our own yet.

Engine support ⬜

As of 2026-09-10 no released engine serves deepseek_v41. vLLM has open PRs (#56214, #56228, #56227, #56208) filed the morning the model dropped, and SGLang has merged a cookbook plus initial support. Serving instructions land here once we have actually served it.

Provenance

Built with dsv41_fp4_stream.py, a shard-streaming re-packer that never holds more than one shard in RAM. The E2M1 table and nibble order are taken verbatim from the checkpoint's own inference/convert.py, not reconstructed from the spec.

License: MIT, inherited from the base model.