---
license: apache-2.0
language:
  - en
  - zh
base_model:
  - Qwen/Qwen3-35B-A3B
  - huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated
  - sakamakismile/Huihui-Qwen3.6-35B-A3B-abliterated-NVFP4
library_name: vllm
pipeline_tag: text-generation
tags:
  - qwen3
  - moe
  - abliterated
  - uncensored
  - nvfp4
  - mtp
  - speculative-decoding
  - blackwell
  - dgx-spark
  - gb10
---

# Qwen3.6-35B-A3B-abliterated-NVFP4-MTP

The **fastest abliterated Qwen3.6 for NVIDIA Blackwell / DGX Spark** — NVFP4 quantized with MTP (Multi-Token Prediction) layers for speculative decoding.

**95.2 tok/s** decode on a single DGX Spark (GB10) with MTP=3.

## What Makes This Different

No other model on HuggingFace combines all three:

| Feature | This Model | sakamakismile NVFP4 | RedHatAI NVFP4 |
|---------|-----------|-------------------|----------------|
| NVFP4 weights | ✅ | ✅ | ✅ |
| MTP layers | ✅ BF16 | ❌ | ✅ |
| Abliterated | ✅ | ✅ | ❌ |
| **Speculative decode** | **✅ 95 tok/s** | **❌ ~70 tok/s** | **✅ but censored** |

Without MTP layers, you're limited to ~70 tok/s on GB10. MTP enables speculative decoding which gives a **+35% speedup**.

## Model Details

- **Architecture**: Qwen3 MoE (Mixture of Experts), 35B total / 3B active parameters
- **Main weights**: NVFP4 quantized (21.9 GB) — optimized for NVIDIA Blackwell CUTLASS kernels
- **MTP layers**: BF16 (1.6 GB, 19 tensors in `model-mtp.safetensors`)
- **Total size**: ~23.5 GB
- **Context length**: 131,072 tokens
- **Abliteration**: Refusal direction removed via representation engineering (from [huihui-ai](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated))

## How It Was Made

1. Started with [sakamakismile/Huihui-Qwen3.6-35B-A3B-abliterated-NVFP4](https://huggingface.co/sakamakismile/Huihui-Qwen3.6-35B-A3B-abliterated-NVFP4) (NVFP4 main weights, no MTP)
2. Extracted BF16 MTP layers from [huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated) (full BF16 model)
3. Injected MTP tensors into a separate `model-mtp.safetensors` file
4. Updated `model.safetensors.index.json` with MTP weight mappings
5. Verified with vLLM speculative decoding (MTP=1 through MTP=4)

## Benchmark Results

**Hardware**: NVIDIA DGX Spark (GB10 Blackwell, sm_121, ~128GB unified memory)
**Engine**: vLLM v0.23.1 with FlashInfer + CUTLASS NVFP4 MoE kernels
**Benchmark**: 256 generated tokens at various context lengths

| Config | 1K ctx | 4K ctx | 16K ctx | 32K ctx | 64K ctx |
|--------|--------|--------|---------|---------|---------|
| **MTP=3, seqs=1 (best)** | **92.2** | **95.2** | **90.7** | **89.7** | **84.4** |
| MTP=2, seqs=4 | 87.2 | 75.5 | 82.8 | 81.6 | 74.4 |
| MTP=4, seqs=1 | 75.6 | 71.3 | 74.0 | 77.6 | 74.4 |
| No MTP (NVFP4 only) | ~71 | ~70 | ~74 | ~68 | ~70 |

MTP=3 is the sweet spot. MTP=4 regresses due to speculation overhead exceeding acceptance rate.

## Recommended vLLM Launch Config

```bash
docker run -d --gpus all --network host --ipc host --shm-size=16g \
  --name vllm-qwen36 \
  -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \
  -e TORCH_MATMUL_PRECISION=high \
  -v /path/to/models:/models \
  ghcr.io/spark-arena/dgx-vllm-eugr-nightly:latest \
  vllm serve /models/Qwen3.6-35B-A3B-abliterated-NVFP4-MTP \
    --host 0.0.0.0 --port 8000 --max-model-len 131072 \
    --max-num-batched-tokens 32768 --max-num-seqs 1 \
    --trust-remote-code --gpu-memory-utilization 0.7 \
    --reasoning-parser qwen3 --kv-cache-dtype fp8 \
    --load-format instanttensor --attention-backend flashinfer \
    --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
    --enable-prefix-caching -tp 1