File size: 1,864 Bytes
dfcd689 0ad7f50 dfcd689 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
base_model: Qwen/Qwen3.6-27B
tags:
- compressed-tensors
- 4-bit
- dense
- deltanet
- thinking
- vision
- multimodal
- rdna4
- rocm
- sglang
- quantized
license: apache-2.0
---
# Qwen3.6-27B AWQ 4-bit (compressed-tensors)
Compressed-tensors output of GPTQ calibration of [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) with thinking + vision preserved.
## Which variant should I download?
| Stack | Recommended | Why |
|---|---|---|
| **SGLang + ROCm** | [Native AWQ](https://huggingface.co/mattbucci/Qwen3.6-27B-AWQ) | Faster on the fused Triton AWQ GEMM than the ROCm CT MoE path |
| **SGLang + NVIDIA** | [Native AWQ](https://huggingface.co/mattbucci/Qwen3.6-27B-AWQ) | Avoids the same `Qwen3_5Moe` CT loader bug seen on 35B (not yet confirmed on 27B but same code path) |
| **vLLM / autoawq / TGI on NVIDIA** | Either works | CT loaders in those engines handle the gate correctly |
| **Inspection / re-conversion** | This (CT) | Raw GPTQ output from llmcompressor before AWQ repack |
## Model Details
| | |
|---|---|
| **Base model** | [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) |
| **Architecture** | Qwen3.5 dense+DeltaNet hybrid + vision tower |
| **Parameters** | 27B |
| **Format** | compressed-tensors `pack-quantized` (W4A16, group_size=128) |
| **Calibration** | GPTQ via llmcompressor, 256 samples × 1024 tokens, `thinking_vision` recipe |
For ignore list and benchmark numbers see the [native variant's README](https://huggingface.co/mattbucci/Qwen3.6-27B-AWQ).
## Convert to native AWQ
```bash
git clone https://github.com/mattbucci/2x-R9700-RDNA4-GFX1201-sglang-inference
python scripts/quantize/convert_moe_ct_to_awq.py <local_path_to_this_repo> <output_dir> --group-size 128
```
## Hardware origin
Calibrated on 2× AMD Radeon AI PRO R9700 (gfx1201, RDNA4) with ROCm 7.2 + SGLang v0.5.10 + RDNA4 patches.
|