---
license: apache-2.0
library_name: transformers
base_model:
  - Kwaipilot/KAT-Coder-V2.5-Dev
inference: false
tags:
  - dflash2
  - speculative-decoding
  - draft-model
  - moe
  - code
  - speculators
  - vllm
---

# KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B

A Mixture-of-Experts DFlash2 drafter for [`Kwaipilot/KAT-Coder-V2.5-Dev`](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev).

Modern MoE target models are well suited to expert offloading on memory-constrained systems and large-scale expert parallelism at datacenter scale. Applying the same sparse structure to speculative drafters is therefore a natural direction. Strong precedents include [`DeepSeek-V3`](https://huggingface.co/deepseek-ai/DeepSeek-V3) MTP and [`DeepSeek-V4-Flash-0731`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731) DSpark.

This release explores the same direction with a DFlash2 drafter: approximately 2.6B total parameters with approximately 0.3B activated per token.

## Results

| Setting               | Value                                                               |
| --------------------- | ------------------------------------------------------------------- |
| Engine                | vLLM                                                                |
| Hardware              | NVIDIA H200, TP1                                                    |
| Concurrency           | 1                                                                   |
| Target                | `Kwaipilot/KAT-Coder-V2.5-Dev` BF16                                 |
| Thinking sampling     | `temperature=1.0`, `top_p=0.95`, `top_k=20`, `presence_penalty=1.5` |
| Thinking mode         | `enable_thinking=true`                                              |
| Non-thinking sampling | `temperature=0.7`, `top_p=0.8`, `top_k=20`, `presence_penalty=1.5`  |
| Non-thinking mode     | `enable_thinking=false`                                             |
| Max new tokens        | 4,096                                                               |
| Context               | native 262,144                                                      |
| Draft tokens          | 7 per verification step                                             |

| Benchmark                  | Samples | Thinking AL | Non-thinking AL |
| -------------------------- | ------: | ----------: | --------------: |
| AA-LCR                     |     100 |       2.8870 |           3.3894 |
| AIME26                     |      30 |       2.8959 |           4.1591 |
| GSM8K                      |   1,319 |       4.2453 |           5.2930 |
| HumanEval                  |     164 |       4.0568 |           5.9836 |
| LiveCodeBench              |   1,055 |       2.7090 |           3.5688 |
| LongBench-v2               |     402 |       2.4484 |           2.8255 |
| MATH500                    |     500 |       3.6183 |           4.7679 |
| MBPP                       |     257 |       3.6543 |           4.5800 |
| MT-Bench                   |      80 |       2.5998 |           2.8009 |
| SWE-bench Pro              |     731 |       3.1425 |           3.4428 |
| SpeedBench / coding        |      80 |       3.3456 |           4.0887 |
| SpeedBench / humanities    |      80 |       2.2735 |           2.7807 |
| SpeedBench / math          |      80 |       2.4900 |           3.3639 |
| SpeedBench / multilingual  |      80 |       3.0767 |           3.0240 |
| SpeedBench / qa            |      80 |       2.5138 |           2.4313 |
| SpeedBench / rag           |      80 |       3.3953 |           3.5579 |
| SpeedBench / reasoning     |      80 |       2.8214 |           3.1751 |
| SpeedBench / roleplay      |      80 |       1.8487 |           1.8848 |
| SpeedBench / stem          |      80 |       2.3774 |           3.0061 |
| SpeedBench / summarization |      80 |       3.0177 |           2.6967 |
| SpeedBench / writing       |      80 |       2.2223 |           2.3711 |

Accepted length is calculated from the raw server counters:

```text
accepted_length = 1 + accepted_tokens / draft_calls
```

## Architecture

| Setting | Value |
| --- | --- |
| Draft layers | 3 |
| Auxiliary hidden-state IDs | `[3, 12, 21, 30, 39]` |
| Hidden size | 2,048 |
| MoE FFN | 256 routed experts, top-8 routing |
| Routed expert intermediate | 512 |
| Shared expert | 1 expert, intermediate size 512 |
| Router | softmax, top-k normalization |
| Shared-expert gate | sigmoid |
| Attention | 32 Q heads / 4 KV heads |
| Head dimension | 128 |
| Attention window | sliding window 2,048 |
| Draft attention | non-causal inside the draft block |
| RoPE theta | 1e7 |
| Norm / activation | RMSNorm eps 1e-6 / SiLU |
| Dynamic convolution | kernel 2, group 16 |
| Candidate selector | rank 256, top-k 16 |
| Block | 7 draft tokens, `sample_from_anchor=true` |
| Draft vocabulary | 248,320 |
| Parameters | 2,638,551,808 total / 298,130,176 activated per token |

## Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with KAT-Coder or a compatible quantized version:

```bash
vllm serve Kwaipilot/KAT-Coder-V2.5-Dev \
  --tensor-parallel-size 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --language-model-only \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B",
    "num_speculative_tokens": 7
  }'
```

This checkpoint currently requires a vLLM build containing both [vLLM PR #54154](https://github.com/vllm-project/vllm/pull/54154), which adds the `sample_from_anchor=true` DFlash query layout, and [vLLM PR #54475](https://github.com/vllm-project/vllm/pull/54475), which adds MoE DFlash2.

## Checkpoint

The BF16 checkpoint contains 63 tensors and 2,638,551,808 parameters.

| File | SHA256 |
| --- | --- |
| `model.safetensors` | `8e96fdfa924405443b566be3a961eb087cd6991665040b05d7c3ec739d5469ee` |

## Future work

- Adaptive verification: explore confidence-head-driven adaptive draft budgets for high-concurrency serving.

## Acknowledgements

This work builds on [`Kwaipilot/KAT-Coder-V2.5-Dev`](https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev), [`vllm-project/speculators`](https://github.com/vllm-project/speculators), and [`vllm-project/vllm`](https://github.com/vllm-project/vllm).

## License

Apache-2.0.