zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4

Qwen3.8-27B-Dominatrix-abliterated — MTP-NVFP4 (DFlash 2 compatible)

Mixed-precision NVFP4/FP8 PTQ of zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated — allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.

23 GB, built with NVIDIA TensorRT Model Optimizer for Blackwell (SM120) inference in SGLang or vLLM.

The distinguishing feature: lm_head is left dense BF16, which is a hard requirement for DFlash 2 speculative decoding. Most ModelOpt NVFP4 exports of this architecture quantize lm_head and therefore cannot run DFlash 2 at all. Cost of the dense head is ~1.9 GB of VRAM over a packed one.

---

Quantization layout

component	precision
MLP gate_proj / up_proj / down_proj	NVFP4 W4A4
self_attn q/k/v/o, linear_attn projections	FP8 e4m3
KV cache	FP8
lm_head	BF16, dense
embed_tokens, MTP head, vision tower	BF16

Export format is ModelOpt MIXED_PRECISION with a per-layer map in hf_quant_config.json. Calibrated on in-domain ChatML roleplay text rather than a generic news corpus.

hf_quant_config.json records producer.version: 0.0.0 because it was built from an editable install. That field is not meaningful provenance.

---

Serving

SGLang with DFlash 2

Requires the z-lab/Qwen3.8-27B-DFlash2 drafter and an SGLang build including PR #35371.

sglang serve \
  --trust-remote-code \
  --model-path /models/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 \
  --mem-fraction-static 0.70 \
  --attention-backend flashinfer \
  --chunked-prefill-size 2048 \
  --reasoning-parser qwen3 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path /models/Qwen3.8-27B-DFlash2-zlab \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant

--kv-cache-dtype can be omitted: this checkpoint declares kv_cache_quant_algo: FP8, so SGLang's auto resolves it.

Without speculative decoding

Drop the four --speculative-* flags. The dense lm_head buys nothing in that configuration but is otherwise harmless.

MTP

The mtp.* tensors survive the quant in BF16, so MTP speculation remains available as an alternative drafter. Pick one — MTP or DFlash 2, not both.

Hardware

NVFP4 requires Blackwell (SM120+) for native FP4 tensor-core execution. Built and tested on an RTX PRO 6000 Blackwell. Weights are ~23 GB, leaving room for a long-context KV cache and the 2B DFlash 2 drafter.

Quality

Fidelity of the underlying BF16 abliteration versus stock Dominatrix is summarised on the BF16 card.

The quantization error of this build has not been measured. Treat it as unquantified. Leaving lm_head dense should help, since the output projection is among the most quantization-sensitive layers, but that is reasoning, not a measurement.

Sampler guidance from upstream Dominatrix carries over: temperature 1.0–1.25 with min_p 0.1 or top_p 0.95; some prefer 0.7 and nothing else.