---
license: apache-2.0
base_model: gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090
pretty_name: Qwen3.8-27B-NVFP4-RTX5090-No-MTP
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - nvfp4
  - modelopt
  - nvidia
  - rtx-5090
  - blackwell
  - quantization
  - qwen3.8
  - vllm
  - sglang
---

> ## ⚠️ Superseded
>
> The main checkpoint now ships with the MTP head removed **and** an NVFP4 `lm_head` — it is strictly better than this variant (17.92 GB, 2 shards).
>
> **Use [`Qwen3.8-27B-NVFP4-RTX5090`](https://huggingface.co/gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090) instead.** This repo stays up so existing pins keep working.

---


# Qwen3.8-27B-NVFP4-RTX5090-No-MTP

[`Qwen3.8-27B-NVFP4-RTX5090`](https://huggingface.co/gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090) with the native MTP draft head removed. **The target model is bit-identical** — same NVFP4 weights, same calibration, same chat template. Only the 15 `mtp.*` tensors are gone.

| | Parent | **This variant** |
| --- | --- | --- |
| Download | 20.59 GB | **19.74 GB** (−0.85 GB, −4.1%) |
| Shards | 3 | **2** |
| Weights in VRAM | 18.80 GB | **18.80 GB** (identical) |
| Speculation on SGLang | DSpark ✅ · MTP ✅ | **DSpark ✅** |
| Speculation on vLLM | MTP ✅ | **none** |

---

## Read this before choosing it

**It saves download size, not memory.** Measured on an RTX 5090, SGLang reports `mem usage=18.80 GB` for weights with and without the head — byte-for-byte the same, because `mtp.*` is namespaced outside the target graph and is never allocated unless you turn speculation on. If you are hoping to free VRAM, this variant will not do that.

**It removes speculative decoding on vLLM entirely.** DSpark is a SGLang-only algorithm, so on vLLM the native MTP head is the *only* draft path. Without it you decode at ~81.6 tok/s instead of ~136.9.

So this variant makes sense if you are SGLang-only and using the [DSpark drafter](https://huggingface.co/gittensor-model-hub/Qwen3.8-27B-DSpark-NVFP4) (which is both faster and smaller than MTP anyway), or if you are shipping the weights somewhere that 0.85 GB matters. **Otherwise use the parent** — the head is free when idle.

### Which drafter should I use anyway?

Even on the parent checkpoint, the MTP head is not the best option:

| Drafter | Decode | Accept | Size |
| --- | --- | --- | --- |
| [**DSpark-NVFP4 v2**](https://huggingface.co/gittensor-model-hub/Qwen3.8-27B-DSpark-NVFP4) (SGLang) | **155.8 tok/s** | **2.886** | **1.41 GB** |
| Native MTP head (SGLang or vLLM) | 136.9 tok/s | 2.758 | 5.53 GB |
| No speculation | 81.6 tok/s | — | — |

DSpark wins on speed, acceptance *and* size — the MTP head's value is that it works on vLLM, where DSpark does not exist.

---

## Serve

SGLang with the DSpark drafter (recommended for this variant):

```bash
sglang serve --model-path gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090-No-MTP \
  --trust-remote-code --tp-size 1 \
  --context-length 65536 --kv-cache-dtype fp8_e4m3 \
  --attention-backend flashinfer --chunked-prefill-size 2048 \
  --mamba-radix-cache-strategy extra_buffer_lazy --mamba-ssm-dtype bfloat16 \
  --mem-fraction-static 0.90 --max-running-requests 2 \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path gittensor-model-hub/Qwen3.8-27B-DSpark-NVFP4 \
  --speculative-dspark-block-size 7 --speculative-draft-model-quantization modelopt_fp4 \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder
```

vLLM for the full 262,144-token window (no speculation available on this variant):

```bash
vllm serve gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090-No-MTP \
  --quantization modelopt --kv-cache-dtype fp8 --trust-remote-code \
  --max-model-len 262144 --max-num-seqs 16 --gpu-memory-utilization 0.97 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml
```

Do **not** pass `--speculative-config '{"method":"mtp"}'` to this variant — the head is not present.

---

## What changed, exactly

Removed 15 tensors, all under the `mtp.` prefix:

```
mtp.fc.weight                     mtp.layers.0.self_attn.{q,k,v,o}_proj.weight
mtp.norm.weight                   mtp.layers.0.self_attn.{q,k}_norm.weight
mtp.pre_fc_norm_embedding.weight  mtp.layers.0.mlp.{gate,up,down}_proj.weight
mtp.pre_fc_norm_hidden.weight     mtp.layers.0.{input,post_attention}_layernorm.weight
```

They were BF16, never quantized (the parent's `hf_quant_config.json` already excluded `mtp*`). Alongside the tensor removal:

- `config.json` — `text_config.mtp_num_hidden_layers` `1 → 0`; `mtp_use_dedicated_embeddings` dropped
- `hf_quant_config.json` — `exclude_modules` 149 → 147 (`mtp*`, `mtp.layers.0*` removed)
- re-sharded 3 → 2 (the third shard held only `mtp.*` and became empty), index and `crc32.txt` regenerated

Everything else — tokenizer, chat template, preprocessor configs, generation config — is copied unchanged from the parent.

Verified: loads in SGLang as `Qwen3_5ForConditionalGeneration`, `quant_algo=NVFP4`, and answers correctly.

---

## Accuracy, context and recipe

Unchanged from the parent, since the target weights are identical. See the [parent model card](https://huggingface.co/gittensor-model-hub/Qwen3.8-27B-NVFP4-RTX5090) for the full quantization recipe, 262K context behaviour, accuracy smoke and chat-template documentation.

## License

Apache 2.0, same as the parent and the Qwen3.8-27B base model.