heath0xFF/DeepSeek-V4-Flash-0731-REAP-K216-GGUF

DeepSeek V4 Flash 0731 REAP K216 GGUF

An experimental, single-DGX-Spark build of DeepSeek V4 Flash 0731 that retains 216 of 256 routed experts per MoE layer.

This repository uses a different construction order from earlier K160 conversions:

1. Start from Unsloth's stock DeepSeek V4 Flash 0731 GGUF quantization.
2. Apply 0xSero's K216 REAP expert map directly to the already-quantized GGUF.
3. Copy every retained quantized expert row byte-for-byte; do not dequantize or requantize it.
4. Replace the compact routing tables for layers 0-2 with the exact tables carried by 0xSero's checkpoint.

This preserves the stock quantization exactly for all retained tensors and avoids quantizing an already transformed/repacked REAP checkpoint.

Files

Variant	Size	Status
DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf	82.94 GiB total	Validated on NVIDIA GB10

The first shard is the entry point; llama.cpp discovers the remaining shards automatically.

Provenance

• Original model: deepseek-ai/DeepSeek-V4-Flash-0731
• Stock GGUF quant: unsloth/DeepSeek-V4-Flash-0731-GGUF, UD-IQ3_XXS
• K216 REAP map: 0xSero/deepseek-v4-flash-0731-spark, revision 22f28d32b9b29b4352eaa380ff8c2c170b2847ab

The included structured report and plan record the selected experts. The included validator compared all 1,328 output tensors and all 89,060,075,612 tensor-data bytes against the stock quant and K216 routing source:

• 1,113 unchanged tensors matched the stock GGUF byte-for-byte;
• 212 compacted expert/router tensors matched their selected stock rows byte-for-byte;
• 3 tid2eid routing tables matched 0xSero's carried tensors exactly.

GB10 test result

Tested on an NVIDIA DGX Spark / GB10 with llama.cpp b10223 (11924d4c1), CUDA architecture 121a.

• Full CUDA offload completed successfully.
• Prompt processing: 33.2 tokens/s in the short smoke test.
• Generation: 15.9 tokens/s in the short smoke test.
• Exact-shape MUL_MAT_ID correctness tests passed for the IQ2_XS gate/up projections and IQ3_XXS down projections with 216 experts and top-6 routing.

Configured-context smoke tests used one slot, Q8_0 K/V cache, full CUDA offload, -b 1024 -ub 512, and the same short non-thinking API prompt:

Configured context	Prompt	Generation	CUDA allocation	Result
32,768	46.8 tok/s	16.8 tok/s	85,192 MiB	Pass
65,536	48.1 tok/s	16.6 tok/s	85,330 MiB	Pass
131,072	44.1 tok/s	16.3 tok/s	85,640 MiB	Pass
262,144	—	—	86,330 MiB	Initializes; first prompt crashes in the current llama.cpp CUDA path

Recommended GB10 context: 131,072 tokens. This is the largest configuration currently validated to initialize and generate successfully. These are short-prompt allocation/generation smoke tests, not tests with the entire context window occupied; sustained long-context quality and memory behavior still need evaluation.

llama.cpp example

Use a recent llama.cpp build with deepseek4 support. On GB10, mmap is recommended; --no-mmap needlessly duplicates memory pressure for this model.

llama-cli \
  -m DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf \
  -ngl 999 \
  --flash-attn on \
  -c 131072 \
  -b 1024 \
  -ub 512 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --temp 1.0 \
  --top-p 1.0 \
  --top-k 0 \
  --min-p 0 \
  --repeat-penalty 1.0 \
  -n 256 \
  -p "Write one sentence about the moon."

DeepSeek V4 Flash defaults to a thinking-style template. For direct-answer diagnostics, disable reasoning when your llama.cpp client or API exposes that option. Do not treat a repetition penalty as a fix for a conversion or template problem.

Reproducibility

• tools/reap_k216_gguf.py performs the byte-preserving rewrite.
• tools/validate_reap_k216_gguf.py performs exhaustive tensor-data validation.
• metadata/REAP_K216_PLAN.json records the rewrite plan.
• metadata/REAP_K216_STRUCTURED_REPORT.json records 0xSero's structured K216 selection.

This is frontier/experimental work. Please include your llama.cpp commit, backend, hardware, context, exact command, and minimal prompt when reporting results.

License and acknowledgements

MIT, inherited from the source model.

Thanks to DeepSeek, Unsloth, 0xSero, the REAP authors, and llama.cpp contributors.