Getting Muse Glimmer 30B + DFlash working on RTX 5090 (Windows) — need help reaching advertised speed

Hi all,

I've been trying to reproduce the 233 tok/s DFlash benchmark for Muse Glimmer 30B K-Quant on an RTX 5090 (32 GB GDDR7). I'm seeing DFlash activate but getting inconsistent results and would love input from anyone who's hit the advertised numbers.

Hardware / Software

Launch command

llama-server.exe `
  --model  "C:\models\Muse-Glimmer-30B\muse-glimmer-30B-kquant-17gb.gguf" `
  --model-draft "C:\models\Muse-Glimmer-30B\dflash-kquant.gguf" `
  --mmproj "C:\models\Muse-Glimmer-30B\mmproj-kquant.gguf" `
  -ngl 99 -ngld 99 `
  --np 1 `
  --ctx-size 131072 `
  --spec-type draft-dflash `
  --spec-draft-n-max 15 `
  --flash-attn on `
  --reasoning-preserve `
  --host 0.0.0.0 --port 8000

Startup log confirms DFlash is loading:

I common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
I common_speculative_impl_draft_dflash: - n_max=15, n_min=0, p_min=0.00
I common_speculative_impl_draft_dflash: - block_size=16, mask_token_id=201818, n_extract=5
I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 131072, kv_unified = 'false'
I srv  llama_server: model loaded

Results

I sent the same coding task (BST implementation, ~80K-token context) multiple times and watched the draft acceptance line in the server logs.

• Prefill rate is very healthy: ~2,800–4,400 tok/s depending on context length.
• Baseline (no draft model): ~74–77 tok/s — consistent with Meta's published 74.9 tok/s baseline.
• Peak decode so far: 168 tok/s (run 1 above) — roughly 2.2× speedup, DFlash is clearly activating.
• Target: 233 tok/s from the model card. Some community posts are quoting 300 tok/s — I don't know the source of that number.

What I've tried

• ✅ --spec-type draft-dflash (was silently defaulting to draft-simple before — this was the key missing flag)
• ✅ --flash-attn on (required for DFlash block-parallel attention)
• ✅ --spec-draft-n-max 15 (block_size=16, trained max = 15)
• ✅ Single slot (--np 1) to avoid splitting KV cache budget
• ❓ kv_unified = 'false' shown in logs — should this be true for DFlash?
• ❓ Is kv_cache_type relevant? (q8_0 / q4_0 for KV compression)
• ❓ Temperature / sampling parameters affecting acceptance rate?
• ❓ Does this need a Linux build to work properly, or is the Windows Clang binary sufficient for DFlash's CUDA kernels?

Questions for the community

1. What acceptance rate should I expect? My run 1 got 5.58 mean accepted tokens per block. Is there a warm-up or context-length dependency? The acceptance drops sharply on follow-up turns.

2. Is kv_unified important for DFlash? The log shows kv_unified = 'false'. DFlash shares KV state between main and draft models — does this flag need to be true?

3. Is there a minimum context length for DFlash to engage properly? Run 1 had an 80K-token prefill; runs 2–3 were shorter continuation turns and acceptance cratered.

4. Are there temperature / sampling settings that improve acceptance? Lower temperature → more predictable outputs → higher draft acceptance?

5. Windows vs. Linux? Are there known issues with the DFlash CUDA kernels in the Windows pre-built binaries? Would a native Linux (WSL2) build be meaningfully faster?

6. Has anyone consistently hit 200+ tok/s on a 5090? If so, please share your full startup command and llama.cpp build.

Thanks in advance — this model is clearly capable, just trying to squeeze every token out of it.