Add Qwen3.5-9B-DFlash release

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,126 @@
+---
+pipeline_tag: text-generation
+library_name: transformers
+base_model:
+  - Qwen/Qwen3.5-9B
+license: apache-2.0
+inference: false
+tags:
+  - dflash
+  - speculative-decoding
+  - speculative-decoding-draft
+  - block-diffusion
+  - draft-model
+  - diffusion-language-model
+  - efficiency
+  - qwen
+  - qwen3
+  - qwen3.5
+  - sglang
+---
+
+# Qwen3.5-9B-DFlash
+
+[Paper](https://arxiv.org/abs/2602.06036) | [Github](https://github.com/z-lab/dflash) | [Blog](https://z-lab.ai/projects/dflash)
+
+This DFlash draft model is a joint retrain from [Z-Lab](https://z-lab.ai) and [Modal](https://modal.com), trained with 40k sequence length and sliding-window attention for improved long-context performance. It is mirrored across the following Hugging Face repositories:
+
+- [`z-lab/Qwen3.5-9B-DFlash`](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash)
+- [`modal-labs/Qwen3.5-9B-DFlash`](https://huggingface.co/modal-labs/Qwen3.5-9B-DFlash)
+
+This repository contains a DFlash draft model for `Qwen/Qwen3.5-9B`. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.
+
+DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.
+
+## Quick Start
+
+This model should be used with an inference server that supports DFlash speculative decoding. An example SGLang deployment is:
+
+```bash
+export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1
+
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3.5-9B \
+  --trust-remote-code \
+  --speculative-algorithm DFLASH \
+  --speculative-draft-model-path modal-labs/Qwen3.5-9B-DFlash \
+  --speculative-dflash-block-size 8 \
+  --speculative-draft-attention-backend fa4 \
+  --attention-backend trtllm_mha \
+  --linear-attn-prefill-backend flashinfer \
+  --linear-attn-decode-backend flashinfer \
+  --mamba-scheduler-strategy extra_buffer \
+  --tp-size 1 \
+  --max-running-requests 32 \
+  --cuda-graph-max-bs-decode 32 \
+  --cuda-graph-backend-prefill tc_piecewise \
+  --enable-flashinfer-allreduce-fusion \
+  --mem-fraction-static 0.8 \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+Block size `8` is the recommended default for higher-concurrency serving. Block size `16` gives longer accept lengths and strong concurrency-1 throughput in most workloads.
+
+## Benchmark Results
+
+We benchmarked DFlash against the autoregressive baseline and Qwen's built-in MTP draft path. DFlash reaches up to `5.01x` speedup at concurrency 1 and `2.58x` at concurrency 32. Across the benchmark suite, DFlash delivers higher throughput than MTP at every matched setting where both completed.
+
+### Setup
+
+- Runtime: SGLang on 1x NVIDIA B200 GPU, tensor parallel size 1, `bfloat16`
+- Backends: `trtllm_mha` target attention, `fa4` DFlash draft attention, `flashinfer` linear-attention prefill and decode
+- Workloads: GSM8K, MATH500, HumanEval, MBPP, and MT-Bench with the Qwen chat template
+- Decoding: greedy, thinking enabled, max output length 4096 tokens
+- Measurement: 5 independent runs per configuration at concurrency 1 and 32 with continuous batching
+- Throughput: generated output tokens / wall-clock benchmark time, including prefill and scheduling
+- Accept length: `completion_tokens / spec_verify_ct` per generation turn, averaged across generation turns
+
+### Throughput and Speedup
+
+Each cell is `output tok/s (speedup)`. Bold marks the fastest speculative configuration in each row.
+
+#### Concurrency 1
+
+| Workload | Baseline | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| gsm8k | 245.2 (1.00x) | 537.7 (2.19x) | 609.8 (2.49x) | 573.4 (2.34x) | 890.7 (3.63x) | 435.5 (1.78x) | **1027.3 (4.19x)** |
+| math500 | 244.6 (1.00x) | 558.3 (2.28x) | 636.3 (2.60x) | 617.3 (2.52x) | 987.6 (4.04x) | 485.3 (1.98x) | **1225.7 (5.01x)** |
+| humaneval | 243.4 (1.00x) | 537.6 (2.21x) | 633.2 (2.60x) | 586.9 (2.41x) | 959.5 (3.94x) | 447.6 (1.84x) | **1195.8 (4.91x)** |
+| mbpp | 244.7 (1.00x) | 526.4 (2.15x) | 624.4 (2.55x) | 543.6 (2.22x) | 935.7 (3.82x) | 403.6 (1.65x) | **1092.6 (4.46x)** |
+| mt-bench | 243.7 (1.00x) | 501.7 (2.06x) | 560.2 (2.30x) | 494.6 (2.03x) | 757.5 (3.11x) | 368.0 (1.51x) | **834.3 (3.42x)** |
+
+#### Concurrency 32
+
+| Workload | Baseline | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| gsm8k | 5837.6 (1.00x) | 10421.0 (1.79x) | 11882.9 (2.04x) | 10132.5 (1.74x) | **13718.0 (2.35x)** | 8332.8 (1.43x) | 11019.7 (1.89x) |
+| math500 | 5885.7 (1.00x) | 11124.9 (1.89x) | 12534.0 (2.13x) | 11213.3 (1.91x) | **15198.3 (2.58x)** | 7785.0 (1.32x) | 13227.4 (2.25x) |
+| humaneval | 5513.0 (1.00x) | 9645.9 (1.75x) | 11882.7 (2.16x) | 9701.0 (1.76x) | **14229.9 (2.58x)** | 6901.7 (1.25x) | 12406.1 (2.25x) |
+| mbpp | 5538.8 (1.00x) | 9116.4 (1.65x) | 11561.2 (2.09x) | 8701.6 (1.57x) | **13460.3 (2.43x)** | 6220.5 (1.12x) | 11338.9 (2.05x) |
+| mt-bench | 5491.7 (1.00x) | 9135.0 (1.66x) | 10072.2 (1.83x) | 8436.9 (1.54x) | **10718.2 (1.95x)** | 5917.8 (1.08x) | 8495.7 (1.55x) |
+
+### Accept Length
+
+Mean accept length at concurrency 1. Bold marks the higher value in each matched MTP/DFlash pair.
+
+| Workload | MTP steps=3 | DFlash block=4 | MTP steps=7 | DFlash block=8 | MTP steps=15 | DFlash block=16 |
+| --- | --- | --- | --- | --- | --- | --- |
+| gsm8k | **3.464** | 3.452 | 5.276 | **5.400** | 6.388 | **6.949** |
+| math500 | 3.541 | **3.555** | 5.466 | **5.757** | 6.728 | **7.721** |
+| humaneval | 3.493 | **3.571** | 5.326 | **5.798** | 6.399 | **7.927** |
+| mbpp | 3.338 | **3.454** | 4.790 | **5.376** | 5.508 | **6.820** |
+| mt-bench | **3.229** | 3.193 | 4.551 | **4.606** | 5.438 | **5.716** |
+
+## Citation
+
+If you find DFlash useful, please cite the original paper:
+
+```bibtex
+@article{chen2026dflash,
+  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
+  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
+  journal = {arXiv preprint arXiv:2602.06036},
+  year    = {2026}
+}
+```

config.json

@@ -0,0 +1,59 @@
+{
+  "architectures": [
+    "DFlashDraftModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoModel": "dflash.DFlashDraftModel"
+  },
+  "bos_token_id": null,
+  "dflash_config": {
+    "block_size": 16,
+    "mask_token_id": 248077,
+    "target_layer_ids": [
+      1,
+      5,
+      9,
+      13,
+      17,
+      21,
+      25,
+      29
+    ]
+  },
+  "dtype": "bfloat16",
+  "eos_token_id": 248044,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 262144,
+  "max_window_layers": 6,
+  "model_type": "qwen3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 6,
+  "num_key_value_heads": 8,
+  "num_target_layers": 32,
+  "pad_token_id": null,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.7.0",
+  "use_cache": true,
+  "use_sliding_window": true,
+  "vocab_size": 248320
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a42274b32554f48de1faa0d42824e9c2ceda649c30ae0a731cddf410dd698c7
+size 2583816465