---
license: mit
language:
- en
- zh
tags:
- custom
- int4
- gptq
- quantized
- amd
- rdna4
- rocm
base_model: Qwen/Qwen3-14B
pipeline_tag: text-generation
library_name: custom
---

# Qwen3-14B INT4 Pure (GPTQ + Hadamard)

Pure INT4 quantization of [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) — all layers quantized to INT4.

**Smaller size** — 9.2 GB vs 11 GB for mixed version. Slightly lower quality but still competitive.

## Quality

| Metric | This Model | Mixed INT4/INT8 | llama.cpp Q4_K_M |
|--------|-----------|-----------------|-----------------|
| **Perplexity** (WikiText-2) | 7.787 | 7.692 | 7.715 |

## Performance (AMD Radeon AI PRO R9700)

| Metric | Speed |
|--------|-------|
| Decode (ctx=128) | **62.7 t/s** |
| Prefill (pp512) | **2076 t/s** |
| VRAM | **~8.5 GB** |

## Quantization Details

- **Method**: INT4 asymmetric with Hadamard rotation + GPTQ calibration
- **All layers**: INT4 (no INT8 sensitive layers)
- **Block size**: 32
- **Calibration**: 256 samples × 256 tokens from RedPajama
- **KV cache**: FP8 E4M3 at inference time

## File Format

Custom `.pt` format — requires [rdna4-quant](https://github.com/JohnTDI-cpu/rdna4-quant) engine.

```
embed.pt          — embedding weights (FP16)
layer_000.pt      — layer 0 (quantized weights + scales + metadata)
...
layer_039.pt      — layer 39
final_norm.pt     — final RMSNorm
lm_head.pt        — LM head (FP16)
meta.pt           — quantization metadata
```

## Usage

```bash
git clone https://github.com/JohnTDI-cpu/rdna4-quant
cd rdna4-quant
pip install -r requirements.txt

# Download weights
huggingface-cli download JohnTdi/Qwen3-14B-INT4-Pure-GPTQ --local-dir quantized_v5_pure_int4

# Run inference
python int4_engine_v5.py --quant-dir quantized_v5_pure_int4 --chat

# Or start API server
python api_server.py --quant-dir quantized_v5_pure_int4
```

## Hardware Requirements

- AMD GPU with ROCm 6.x+ support (RDNA3/4, MI300X)
- ~9 GB VRAM
- ROCm 6.x or 7.x, PyTorch 2.6+

## License

MIT — same as the inference engine. Base model (Qwen3-14B) has its own license from Alibaba.