---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:
- en
- zh
tags:
- qwen3_5
- qwen3.8
- nvfp4
- compressed-tensors
- vllm
- blackwell
- rtx-5090
- sm120
- multimodal
- vision
- video
- mtp
- speculative-decoding
- tool-calling
- reasoning
- abliterated
---

# Huihui Qwen3.8-27B Abliterated NVFP4 MTP VL

A **compressed-tensors NVFP4 W4A4** release of [huihui-ai/Huihui-Qwen3.8-27B-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated) at revision `d42ca8978c5a66e92c3446d46e8adfe03ef692ff`. The same-source **BF16 vision/video tower** (333 tensors) and all **15 BF16 MTP tensors** are retained.

## Quick start — RTX 5090 / Blackwell

```bash
hf download lyf/Qwen3.8-27B-Huihui-Abliterated-NVFP4-MTP-VL --local-dir ./huihui-qwen38-nvfp4

docker run --rm --gpus all --ipc=host --network=host   -e VLLM_NVFP4_GEMM_BACKEND=flashinfer-cutlass   -e VLLM_USE_FLASHINFER_SAMPLER=1   -v "$PWD/huihui-qwen38-nvfp4:/model:ro"   vllm/vllm-openai:qwen38-x86_64-cu130   /model   --served-model-name qwen38-huihui-nvfp4   --host 0.0.0.0 --port 8000   --max-model-len 4096   --kv-cache-dtype fp8   --gpu-memory-utilization 0.92   --max-num-seqs 1   --max-num-batched-tokens 1024   --enable-prefix-caching   --trust-remote-code   --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'
```

The command above is the verified multimodal/MTP smoke profile. Larger context values require workload-specific VRAM validation.

## Quantization and lineage

| Component | Format/source |
|---|---|
| Language-model Linear layers | NVFP4 W4A4, group size 16 |
| Vision/video tower | 333 tensors, BF16, same Huihui checkpoint |
| MTP head | 15 tensors, BF16, same Huihui checkpoint |
| `lm_head`, token embedding, GDN `conv1d` | BF16 |
| Calibration | CNN/DailyMail 3.0.0, 20 × 8192 tokens |
| Packaging | compressed-tensors `nvfp4-pack-quantized` |

No official Qwen, Unsloth, Blackfrost, or other behavioral variant weights were grafted into this release.

## Validation

Validated on one RTX 5090, 450 W cap, using `vllm/vllm-openai:qwen38-x86_64-cu130`:

- `GET /health` and `/v1/models`: passed
- OpenAI-compatible 1024-token text generation: passed
- Native MTP n=3: passed
- MTP draft tokens: 1362; accepted: 572; acceptance: 42.0%
- Mean acceptance length: 2.33
- Per-position acceptance: `0.602 / 0.429 / 0.295`
- Client elapsed for 1024 output tokens: 10.73 s
- Runtime VRAM under generation: ~28,944 MiB

The earlier full release workflow also verifies 333 vision tensors and 15 same-source MTP tensors statically. The NInfer derivative in the companion repository carries a separate real-image runtime validation.

## Files

- `model-00001-of-00002.safetensors`, `model-00002-of-00002.safetensors`: compressed checkpoint
- `model-mtp-extra.safetensors`: 15 same-source BF16 MTP tensors
- `model.safetensors.index.json`: complete 2687-tensor index
- `BUILD_MANIFEST.json`, `VALIDATION_REPORT.json`, `STATIC_VALIDATION_REPORT.json`, `recipe.yaml`, `SHA256SUMS`: provenance and reproducibility

## Intended use

This is an abliterated behavioral derivative intended for model research and local inference. Users are responsible for downstream use and applicable policies.