Instructions to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4

SGLang

How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4 with Docker Model Runner:
```
docker model run hf.co/LibertAIDAI/DeepSeek-V4.1-Flash-NVFP4
```

jon1012 commited on 5 days ago

Commit

ae846b7

verified ·

1 Parent(s): f5110b4

Card: verified transcode is bit-exact; credit s-zaizen prior art; Engram is the real differentiator

Browse files

Files changed (1) hide show

README.md +30 -12

README.md CHANGED Viewed

@@ -25,9 +25,8 @@ by [LibertAI](https://libertai.io).
 ## ⚠️ Read this first: V4.1-Flash already ships in 4-bit
-If you came here expecting the usual ~70% NVFP4 cut, **it does not exist for this model,
-and any repo that claims one is measuring against something other than the released
-checkpoint.** DeepSeek quantized V4.1-Flash themselves. From the upstream `config.json`:
 ```json
 "quantization_config": {
@@ -70,10 +69,26 @@ so the identity holds for every block whose scale lands in that window — the g
 is chosen per tensor to centre it. ⬜ Fraction of blocks outside the window: to be
 reported here per tensor (expected 0).
 **This costs size.** NVFP4 is 4 + 8/16 = 4.5 bits/weight against MXFP4's 4 + 8/32 = 4.25,
-so the experts *grow* from 268.9 to ~284.6 GiB. Take this variant only if your engine's
-Blackwell MoE kernels want NVFP4 and have no MX path — it is a compatibility shim, not a
-compression win, and we would rather say so than let you find out from a disk-full error.
 ### 2. Engram: FP8 → FP4, the one real size lever ⬜
@@ -96,12 +111,15 @@ Upstream's own `convert.py` already shards `engram.embed` along dim 0 across ran
 ### Net effect ⬜
-| | as shipped | this repo |
-|---|---:|---:|
-| routed experts + MTP | 276.3 GiB | ~292 GiB |
-| Engram | 189.1 GiB | ~97 GiB |
-| everything else | 8.9 GiB | 8.9 GiB |
-| **total** | **475.2 GiB** | **~398 GiB** |
 ## Hardware notes ⬜

 ## ⚠️ Read this first: V4.1-Flash already ships in 4-bit
+If you came here expecting the usual ~70% NVFP4 cut, **it does not exist for this model.**
+DeepSeek quantized V4.1-Flash themselves. From the upstream `config.json`:
 ```json
 "quantization_config": {
 is chosen per tensor to centre it. ⬜ Fraction of blocks outside the window: to be
 reported here per tensor (expected 0).
+✅ Verified on shard 4: 1,152 expert weights transcoded, and an independent
+reconstruction from both representations found **0 mismatches over 2.36 billion elements,
+`max|delta| = 0` exactly**, with **0 of 849,346,560 block scales** falling outside E4M3's
+window.
 **This costs size.** NVFP4 is 4 + 8/16 = 4.5 bits/weight against MXFP4's 4 + 8/32 = 4.25,
+so the experts *grow*: shard 4 went from 7,389,759,032 to 7,814,559,376 bytes, **+5.75%**.
+Take this variant only if your engine's Blackwell MoE kernels want NVFP4 and have no MX
+path — it is a compatibility shim, not a compression win, and we would rather say so than
+let you find out from a disk-full error.
+To be clear about prior art: this losslessness is not something we invented, it is what
+any careful implementation of this transcode gets, for the structural reason above. We
+checked [`s-zaizen/DeepSeek-V4.1-Flash-NVFP4`](https://huggingface.co/s-zaizen/DeepSeek-V4.1-Flash-NVFP4),
+published the same day, by reconstructing `layers.1.ffn.experts.0.w1` from both their
+checkpoint and the source: **their nibbles are identical and all 11,796,480 elements match
+exactly too.** Their repo is **491.1 GiB — larger than the 475.2 GiB original** — because
+the expert transcode grows the file and the Engram tables are left at FP8. That is the gap
+this repo is trying to close, and it is the only thing that meaningfully separates the
+two.
 ### 2. Engram: FP8 → FP4, the one real size lever ⬜
 ### Net effect ⬜
+| | upstream | s-zaizen NVFP4 | this repo |
+|---|---:|---:|---:|
+| routed experts + MTP | 276.3 GiB | ~292 GiB | ~292 GiB |
+| Engram | 189.1 GiB | 189.1 GiB | ~97 GiB |
+| everything else | 8.9 GiB | 8.9 GiB | 8.9 GiB |
+| **total** | **475.2 GiB** | **491.1 GiB** ✅ | **~398 GiB** ⬜ |
+The Engram column is the whole point. If you need NVFP4 experts and do not care about
+Engram, s-zaizen's repo is a perfectly good checkpoint and was there first.
 ## Hardware notes ⬜