---
base_model: meta-llama/Llama-2-13b-hf
license: llama2
tags:
  - arxiv:2607.07964
  - kronq
  - quantization
  - group-quantization
  - int4
language:
  - en
pipeline_tag: text-generation
---
# Llama-2-13b — KronQ W4A16 g128 (packed int4)

**Paper:** [arXiv:2607.07964](https://arxiv.org/abs/2607.07964) · **Code:** [GitHub](https://github.com/Intelligent-Computing-Lab-Panda/KronQ)

[Llama-2-13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) quantized to **4-bit weights / 16-bit activations**, **group size 128**, with **KronQ** (Kronecker-factored Hessian quantization). Packed int4 (~7.0 GB); a fused dequant + bidirectional-incoherence (BiIP) CUDA kernel with per-group scales unpacks on the fly.

## Results (WikiText-2, seqlen 2048)

**Perplexity:** **4.935**

**Zero-shot accuracy:**

| PIQA | ARC-E | ARC-C | HellaSwag | WinoGrande | BoolQ | OBQA | Average |
|---|---|---|---|---|---|---|---|
| 80.25 | 75.72 | 48.63 | 78.96 | 72.06 | 80.76 | 45.00 | **68.77** |

(lm-evaluation-harness 0-shot; acc_norm for PIQA/HellaSwag/ARC, acc for WinoGrande/BoolQ.)

## Usage

```bash
python eval_pretrained.py meta-llama/Llama-2-13b-hf donghyunli/Llama-2-13b-KronQ-W4A16-g128 --ppl --zs
```

## Recipe

Group-128 asymmetric W4, weight-only (a_bits=16), `--alpha 0.25`, `--act_order`, BiIP, raw H_G. 128 WikiText-2 calibration sequences.

## License

Derivative of Llama-2-13b — [llama2 license](https://ai.meta.com/llama/license/).