---
license: apache-2.0
base_model:
  - deepseek-ai/DeepSeek-V4-Flash-DSpark
tags:
  - vllm
  - dspark
  - speculative-decoding
  - nvfp4
  - kv-cache
  - dgx-spark
  - gb10
  - long-context
  - deepseek-v4
library_name: vllm
pipeline_tag: text-generation
---

# DeepSeek-V4-Flash-DSpark · NVFP4-KV · 1.5M CTX · 3M Pool · C=12 · 2× DGX Spark

A **serving recipe / configuration card** for running
[`deepseek-ai/DeepSeek-V4-Flash-DSpark`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark)
with **DSpark speculative decoding** and an **`nvfp4_ds_mla` 4-bit KV cache** on **2× NVIDIA
DGX Spark (GB10, sm_121a)** at TP=2, for a **1.5M-token context window**.

> **This card does not re-host model weights.** NVFP4 here is a *runtime KV-cache* setting —
> the weights are the unmodified base model. Pull the base model from `deepseek-ai` and apply
> this recipe (below). Full sources, benchmark harness, and the vLLM 0.24.0 port fix live in the
> GitHub repo:
> **https://github.com/drowzeys/keys-vLLm-0.24.0-Optimized-DeepSeekV4-Flash-DSpark-NVFP4-KV-1.5M-CTX-3M-Pool-C-12-on-2-DGX-Spark**

## Headline

- **1.5M context window** backed by a **3,231,736-token KV pool** (util 0.85).
- Coherent needle retrieval validated to **543,994 tokens** — no garble.
- **C=12 concurrency at ~255 tok/s** aggregate; **61–67% DSpark acceptance**.
- Runs on **two DGX Spark boxes** over RoCE/IB.

## Standing config

| Parameter | Value |
|---|---|
| KV cache | `nvfp4_ds_mla` (4-bit MLA KV) |
| Context window | 1,500,000 (`max_model_len`) |
| KV pool | 3,231,736 tokens (21.25 GiB @ util 0.85) |
| `max_num_seqs` | 12 |
| `gpu_memory_utilization` | 0.85 |
| Speculative | DSpark, `MTP_NUM_TOKENS=5` |
| `VLLM_USE_B12X_WO_PROJECTION` | 1 (high-acceptance-under-concurrency profile) |
| Topology | 2× DGX Spark GB10, TP=2 |

Native trained context is 1,048,576; 1M–1.5M is RoPE-extended
(`VLLM_ALLOW_LONG_MAX_MODEL_LEN=1`). Validated-coherent to 512k. See `RESULTS.md`.

## Download & serve (two commands)

```bash
# 1) pull the (unmodified) base model
hf download deepseek-ai/DeepSeek-V4-Flash-DSpark --local-dir ./DeepSeek-V4-Flash-DSpark

# 2) build the GB10 serving image + launch with this recipe
#    (image build + launcher in the GitHub repo; run worker then head)
ssh <worker> '~/dspark-nvfp4-serve.sh 1'
~/dspark-nvfp4-serve.sh 0     # serves an OpenAI-compatible API on :8000
```

`dspark-nvfp4-serve.sh` (included here) carries the exact serve args, the NVFP4/DSpark/garble-fix
env, and the 1.5M/util-0.85/seqs-12/WO=1 profile. Edit the fabric vars (MASTER, IF, HCA, GID)
and model path for your cluster.

## Files

- `dspark-nvfp4-serve.sh` — the serving launcher (this exact config)
- `RESULTS.md` — full benchmarks: concurrency, context, and long-context (6k→512k) sweeps
- `CREDITS.md` — attribution

## Credits

Special thanks to **tonyd2wild** (the `nvfp4_ds_mla` 1M stage recipe), **MiaAI-Lab** (DSpark
DGX Spark packaging), **Rafael Caricio** (DSpark vLLM integration), and the **vLLM** project.
Transplant kernels credited to **aidendle94** (compiled flashinfer sparse-MLA + DeepGEMM),
**CosmicRaisins** (Triton sparse-MLA), and **hazyumps** (GB10 indexer). Model:
**deepseek-ai/DeepSeek-V4-Flash-DSpark**. Full details in `CREDITS.md`.

## Support / Donations

If this work is useful, donations help fund more open GB10 / DGX Spark serving research:

- **Solana:** `drkeys.sol`

Thank you 🙏