Instructions to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
# Run inference directly in the terminal:
llama cli -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
# Run inference directly in the terminal:
llama cli -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
# Run inference directly in the terminal:
./llama-cli -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Use Docker

docker model run hf.co/sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

LM Studio
Jan

vLLM

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Ollama
How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with Ollama:
```
ollama run hf.co/sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
```
Unsloth Desktop

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with Docker Model Runner:
```
docker model run hf.co/sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4
```

Lemonade

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Run and chat with the model

lemonade run user.DeepSeek-v4-Flash-REAP-K150-NVFP4-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

DeepSeek V4 Flash — REAP K150 Mixed NVFP4 (NVFP4 + Q2_K + Q8)

REAP-pruned DeepSeek V4 Flash at K150 (150 routed experts per MoE layer, 41% pruning), with a hybrid mixed-precision quantization targeting NVIDIA DGX Spark (GB10 Blackwell, sm_121).

At a Glance


Base model	DeepSeek V4 Flash
Source checkpoint	deepseek-ai/DeepSeek-V4-Flash (MXFP4/MXFP8 mixed precision)
Pruning method	REAP (Router-weighted Expert Activation Pruning) — Cerebras Research
Routed experts	150 per layer (down from 256)
Hash-preserved	Layers 0-2 (256 experts each — hash-routed layers must remain full-width)
Pruned	Layers 3-42 (150 experts each)
Format	ds4-compact-v2 GGUF with NVFP4 multi-tensor convention
File size	~86 GiB

Mixed Quantization Structure

This model uses a hybrid quant strategy optimized for DGX Spark's bandwidth profile (273 GB/s LPDDR5X, ~97 GB/s managed-memory serving path):

Component	Quant	bpw	BW (GB10)
Routed experts — gate (w1)	NVFP4 (e2m1 + e4m3 per-16 + fp32 scale_2)	4.50	~140 GB/s
Routed experts — up (w3)	NVFP4 (e2m1 + e4m3 per-16 + fp32 scale_2)	4.50	~140 GB/s
Routed experts — down (w2)	Q2_K	2.625	~160 GB/s
Attention (q, k, v, o)	Q8_0	8.5	~228 GB/s
Shared experts (w1, w2, w3)	Q8_0	8.5	~228 GB/s
Output head	Q8_0	8.5	~228 GB/s
Token embeddings	F16	16	~208 GB/s
Norms, HC base/scale	F32	32	~247 GB/s
HC fn weights	F16	16	~208 GB/s

Why this mix?

The bottleneck in MoE decode is gate+up expert projection (~2/3 of expert bytes). Standard GGUF quants use IQ2_XXS here. NVFP4 reads more bytes (4.50 vs 2.06 bpw) but offers substantially better precision at the same bit budget, with dequant bandwidth at ~140 GB/s on GB10. In practice, NVFP4 decode is slightly slower than IQ2_XXS due to the higher byte volume, but the precision gain is well worth the small throughput cost. Down experts stay Q2_K (already saturating at ~160 GB/s). Attention and shared experts stay Q8_0 for quality.

NVFP4: lossless MXFP4→NVFP4 conversion

The HF source checkpoint stores experts in MXFP4 (e2m1 nibbles + e8m0 per-32 block scale). NVFP4 is e2m1 + e4m3 per-16 + per-expert fp32 scale_2. The e2m1 nibbles are identical — the conversion is a scale-only transform. No weight requantization, no quality loss from the quantization step. The NVFP4 weights are bit-identical to the MXFP4 originals; only the scale format changes.

Domain Split (Calibration)

8,000 prompts · 5.0M tokens · calibration on DGX Spark (NVIDIA GB10) at 4,096 token context. REAP activation_energy_sum2 score metric.

Domain	Share
Coding & development	35–40%
Agentic tool-calling	16%
Research & knowledge	15–20%
Math & science	10–15%
Design & planning	5–10%
Trivia & general QA	3–5%

How to Run

⚠️ Standard llama.cpp / GGUF runtimes will NOT load this model. It uses a custom NVFP4 multi-tensor GGUF convention (.nvfp4_weight + .nvfp4_scale_2) plus NVFP4 CUDA kernels, managed-memory serving, and FP8-packed KV cache.

Required Runtime

Clone and build the custom ds4 engine:

git clone https://github.com/sleepyeldrazi/ds4-nvfp4-spark
cd ds4-nvfp4-spark
make

Inference (DGX Spark / NVIDIA GB10)

# Managed memory required — saves ~10 GiB RAM, costs 1-3 t/s
DS4_CUDA_MANAGED_MODEL=1 ./ds4 --cuda -m DeepSeek-V4-Flash-REAP-K150-hybrid.gguf --ctx 131072

# With FP8-packed KV cache
DS4_KV_TURBO=1 DS4_CUDA_MANAGED_MODEL=1 ./ds4 --cuda \
  -m DeepSeek-V4-Flash-REAP-K150-hybrid.gguf --ctx 262144

API Server

DS4_CUDA_MANAGED_MODEL=1 ./ds4-server --cuda -m DeepSeek-V4-Flash-REAP-K150-hybrid.gguf \
  --host 0.0.0.0 --port 17777 --ctx 131072

Memory Budget (DGX Spark, 128 GB unified memory)

Context	K150 resident
Short (8K)	~90 GiB ✅
256K	~99 GiB ✅
1M (FP8 KV)	~113 GiB ✅ (tight)
1M (FP32 KV)	~121 GiB ⚠️ (borderline)

K150 needs managed memory (DS4_CUDA_MANAGED_MODEL=1) to avoid duplication. This saves ~10 GiB of RAM at the cost of 1-3 t/s lower throughput. 1M context requires FP8-packed KV (DS4_KV_TURBO=1).

How It Was Built

Source: deepseek-ai/DeepSeek-V4-Flash official MXFP4/MXFP8 mixed-precision checkpoint
REAP plan: 8,000 prompts → ds4 imatrix on DGX Spark → activation_energy_sum2 per-expert scores → top-150 per layer (layers 3–42), layers 0–2 preserved at 256
NVFP4 repack: gate+up expert e2m1 nibbles copied losslessly from MXFP4; e8m0 per-32 scales converted to e4m3 per-16 + fp32 scale_2; down experts kept as Q2_K
Copy policy: attention, shared experts, output head → Q8_0; embeddings → F16; norms/HC → original precision
Emission: single-pass GGUF generation via deepseek4-quantize (NVFP4 emission, REAP pruning, type-change tracking)

No fine-tuning. Purely structural expert removal + lossless NVFP4 repack. Weights are unmodified — a subset of the original MXFP4 experts.

Variants

Variant	Experts	Size	Best for
K128	128 (50% pruned)	76 GiB	Comfortable fit, 1M ctx
K150 (this)	150 (41% pruned)	86 GiB	Better quality, ~256K ctx
K180	180 (30% pruned)	99 GiB	Best quality, managed memory required

Acknowledgments

DeepSeek — DeepSeek V4 Flash base model
antirez — ds4 engine and GGUF quants
Cerebras Research — REAP expert pruning
NVIDIA — DGX Spark (GB10) hardware
eouya2 — reap-for-ds4 tooling

Downloads last month: 157

GGUF

Model size

178B params

Architecture

deepseek4

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

Base model

deepseek-ai/DeepSeek-V4-Flash

Quantized

(124)

this model

Paper for sleepyeldrazi/DeepSeek-v4-Flash-REAP-K150-NVFP4

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 23