---
license: apache-2.0
base_model: syvai/Qwen3.8-27B-DFlash2-W4A16
base_model_relation: adapter
library_name: vllm
tags:
  - dflash2
  - speculative-decoding
  - draft-model
  - compressed-tensors
  - w4a16
  - int4
  - long-context
  - vllm
  - qwen3
---

# Qwen3.8-27B-DFlash2-W4A16 — long-context config

> [!IMPORTANT]
> **The weights in this repo are byte-identical to
> [`syvai/Qwen3.8-27B-DFlash2-W4A16`](https://huggingface.co/syvai/Qwen3.8-27B-DFlash2-W4A16).**
> Exactly one field in `config.json` differs:
>
> ```diff
> - "max_position_embeddings": 262144
> + "max_position_embeddings": 393216
> ```
>
> That is the whole change. If you are serving at or below 262,144 tokens, use
> the original — this repo gives you nothing. All quantization credit belongs to
> [syv-ai](https://github.com/syv-ai/qwen38-27b-rtx3090), and all model credit to
> [inco.ai / z-lab](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) and **Qwen**.

## Why it exists

The DFlash2 drafter ships `max_position_embeddings: 262144` with plain,
non-YaRN rope. When the **target** is YaRN-extended past its native 262,144
context, the drafter is asked to draft at positions its rope table does not
cover. One token past the limit:

```
Assertion `index out of bounds: 0 <= ... < 262144` failed
  -> torch.AcceleratorError: CUDA error: device-side assert triggered
  -> vllm.v1.engine.exceptions.EngineDeadError
```

This kills the **entire vLLM engine**, not just the offending request — a
mid-session hard failure on any long-context deployment.

## Why raising it is safe

The drafter uses **sliding-window attention (window 2048)**. RoPE affects
attention through *relative* offsets, and those never exceed the window, so
extending the absolute position range does not change what the drafter computes
inside any window. Nothing is retrained and no weight is touched; the rope cache
is simply built large enough to index.

It is not free, though — beyond the drafter's trained range it is extrapolating,
and acceptance degrades:

| target context | acceptance length | result |
|---|---|---|
| 150k | **3.82** | fine |
| 300k, original config | — | **engine dead** |
| 300k, this config | **2.70** | works, coherent output |

Measured greedy, `num_speculative_tokens=7`, on
[`Swift-Qwen3.8-27b-heretic-W8A8-DFlash2`](https://huggingface.co/akumaburn/Swift-Qwen3.8-27b-heretic-W8A8-DFlash2),
one CMP 170HX (64 GB), `fp8_e4m3` KV. At 300k the drafter still beats the
in-checkpoint MTP head (2.51), so DFlash2 remains the better speculation choice
even there.

Set `max_position_embeddings` to *your* served length; 393216 here matches a
1.5× YaRN extension of Qwen3.8-27B's native context.

## Also requires a patched vLLM

Being pack-quantized, this drafter additionally needs the quantized-drafter fix —
stock vLLM 0.29.0 reads `qkv_proj.weight`, which a packed checkpoint does not
have, and fails at engine init with
`AttributeError: 'QKVParallelLinear' object has no attribute 'weight'`
(`qwen3_dflash.py:472`). See
[vllm-project/vllm#51684](https://github.com/vllm-project/vllm/pull/51684)
(comprehensive, upstream) or
[akumaburn/vllm-dflash2](https://github.com/akumaburn/vllm-dflash2) (narrow
stopgap). A BF16 drafter needs no patch.

## Usage

```sh
vllm serve akumaburn/Swift-Qwen3.8-27b-heretic-W8A8-DFlash2 \
  --max-model-len 393216 --kv-cache-dtype fp8_e4m3 --async-scheduling \
  --hf-overrides '{"text_config":{"rope_parameters":{"rope_type":"yarn","factor":1.5,"original_max_position_embeddings":262144}}}' \
  --speculative-config '{"method":"dflash","model":"akumaburn/Qwen3.8-27B-DFlash2-W4A16-longctx","num_speculative_tokens":7}'
```

`num_speculative_tokens=7` is the architectural maximum (`block_size` is 8).
The target must be **unrotated** — against a QuaRot-rotated checkpoint this
drafter rejects every draft (acceptance exactly 1.00).