---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated
base_model_relation: quantized
tags:
- nvfp4
- fp4
- modelopt
- tensorrt
- sglang
- dflash2
- speculative-decoding
- roleplay
- creative-writing
- abliterated
- uncensored
- qwen3
- not-for-all-audiences
---

# Qwen3.8-27B-Dominatrix-abliterated — MTP-NVFP4 (DFlash 2 compatible)

Mixed-precision NVFP4/FP8 PTQ of
**[zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated](https://huggingface.co/zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated)**
— allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.

**23 GB**, built with NVIDIA TensorRT Model Optimizer for Blackwell (SM120) inference in
SGLang or vLLM.

The distinguishing feature: **`lm_head` is left dense BF16**, which is a hard requirement for
DFlash 2 speculative decoding. Most ModelOpt NVFP4 exports of this architecture quantize
`lm_head` and therefore **cannot** run DFlash 2 at all. Cost of the dense head is ~1.9 GB of
VRAM over a packed one.

---

## Quantization layout

| component | precision |
|---|---|
| MLP `gate_proj` / `up_proj` / `down_proj` | **NVFP4** W4A4 |
| `self_attn` q/k/v/o, `linear_attn` projections | **FP8** e4m3 |
| KV cache | FP8 |
| **`lm_head`** | **BF16, dense** |
| `embed_tokens`, MTP head, vision tower | BF16 |

Export format is ModelOpt `MIXED_PRECISION` with a per-layer map in `hf_quant_config.json`.
Calibrated on in-domain ChatML roleplay text rather than a generic news corpus.

> `hf_quant_config.json` records `producer.version: 0.0.0` because it was built from an
> editable install. That field is not meaningful provenance.

---

## Serving

### SGLang with DFlash 2

Requires the [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2)
drafter and an SGLang build including PR #35371.

```bash
sglang serve \
  --trust-remote-code \
  --model-path /models/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 \
  --mem-fraction-static 0.70 \
  --attention-backend flashinfer \
  --chunked-prefill-size 2048 \
  --reasoning-parser qwen3 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path /models/Qwen3.8-27B-DFlash2-zlab \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant
```

`--kv-cache-dtype` can be omitted: this checkpoint declares `kv_cache_quant_algo: FP8`, so
SGLang's `auto` resolves it.

### Without speculative decoding

Drop the four `--speculative-*` flags. The dense `lm_head` buys nothing in that configuration
but is otherwise harmless.

### MTP

The `mtp.*` tensors survive the quant in BF16, so MTP speculation remains available as an
alternative drafter. Pick one — MTP or DFlash 2, not both.

## Hardware

NVFP4 requires **Blackwell (SM120+)** for native FP4 tensor-core execution. Built and tested
on an RTX PRO 6000 Blackwell. Weights are ~23 GB, leaving room for a long-context KV cache and
the 2B DFlash 2 drafter.

## Quality

Fidelity of the underlying BF16 abliteration versus stock Dominatrix is summarised on the
[BF16 card](https://huggingface.co/zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated).

**The quantization error of *this* build has not been measured.** Treat it as unquantified.
Leaving `lm_head` dense should help, since the output projection is among the most
quantization-sensitive layers, but that is reasoning, not a measurement.

Sampler guidance from upstream Dominatrix carries over: temperature **1.0–1.25** with `min_p
0.1` or `top_p 0.95`; some prefer **0.7** and nothing else.