---
license: llama3.2
library_name: qualcomm-ai-runtime
tags:
  - qualcomm
  - qnn
  - htp
  - npu
  - edge-ai
  - z4-quantized
  - llama
base_model: meta-llama/Llama-3.2-1B-Instruct
---

# Llama 3.2 1B Instruct - QNN HTP Z4 Quantized

Pre-compiled model binary for **Qualcomm Hexagon HTP NPU** inference using **QNN GenAI Transformer** backend.

## Model Details

| Property | Value |
|----------|-------|
| Base Model | [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) |
| Quantization | Z4 (Qualcomm 4-bit) |
| Format | QNN GenAI Transformer single binary |
| SDK Version | QAIRT v2.38.0.250901 |
| Target Hardware | Qualcomm Hexagon HTP v73+ NPU |
| Binary Size | ~1.6 GB |
| Tensors | 147 (Z4 for weights, F32 for norms) |

## Tested Hardware

- **Qualcomm IQ-9075 EVK** (QCS9075 SoC, Hexagon HTP v73)

## Usage

This binary is designed for use with the **Qualcomm Genie** runtime (`libGenie.so`) or `genie-t2t-run` CLI.

### With genie-t2t-run

```bash
cd /path/to/model/
LD_LIBRARY_PATH=/path/to/qnn-libs:/usr/lib \
ADSP_LIBRARY_PATH="/usr/lib/dsp/cdsp;/usr/lib/dsp/cdsp1" \
genie-t2t-run -c genie_config.json -p "Hello, how are you?"
```

### genie_config.json

```json
{
  "dialog": {
    "backend": "QnnGenAiTransformer",
    "model-path": "llama3.2-1b-instruct-z4.bin",
    "tokenizer": "tokenizer.json"
  }
}
```

## Compilation

Compiled using QAIRT SDK v2.38.0 GenAI Transformer Composer:
- Source: `meta-llama/Llama-3.2-1B-Instruct` (HuggingFace)
- Quantization: Z4 (4-bit weights, F32 normalization layers)
- Compile time: ~0.3 minutes on x86_64

## License

This model inherits the [Llama 3.2 Community License](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/blob/main/LICENSE).