---
language:
  - en
  - de
  - fr
  - it
  - pt
  - hi
  - es
  - th
license: other
license_name: llama4
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE
tags:
  - gguf
  - llama
  - llama4
  - meta
  - quantized
  - batiai
  - 109b
  - moe
  - multimodal
  - vision
  - frontier
base_model: meta-llama/Llama-4-Scout-17B-16E-Instruct
pipeline_tag: image-text-to-text
library_name: llama.cpp
---

# Llama 4 Scout 17B-16E-Instruct GGUF — Quantized by BatiAI

<p align="center">
  <a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-on--device%20AI-blue?style=for-the-badge&logo=apple" alt="BatiFlow"></a>
  <a href="https://ollama.com/batiai/llama4-scout"><img src="https://img.shields.io/badge/Ollama-batiai%2Fllama4--scout-green?style=for-the-badge" alt="Ollama"></a>
</p>

> imatrix-calibrated GGUF quantizations of **meta-llama/Llama-4-Scout-17B-16E-Instruct** (109B total / 17B active MoE, 16 experts, multimodal).
> Quantized directly from official Meta BF16 weights by [BatiAI](https://bati.ai).

## Why Llama 4 Scout?

- **109B total / 17B active** Mixture-of-Experts (16 experts, top-1 routing) — efficient for size
- **Multimodal native**: text + vision via `mmproj` (image-text-to-text)
- **Multilingual**: 8 official languages + general multilingual
- **Native tool calling** + extended context
- **Meta Llama 4 Community License** — commercial-friendly for most cases (see license link)
- Released **2025-04** by Meta

## Quick Start

```bash
# Q4_K_M (recommended balance, 60GB, M4 Max 128GB ~ M2 Ultra 192GB)
ollama pull batiai/llama4-scout:q4

# IQ3_XXS (smallest, 38GB, M4 Max 64GB+)
ollama pull batiai/llama4-scout:iq3

# Q5_K_M (higher quality, 72GB, M2 Ultra 192GB+)
ollama pull batiai/llama4-scout:q5
```

## Available Quantizations

| Quant | Size | Min RAM | Target Hardware | Notes |
|-------|------|---------|-----------------|-------|
| **IQ3_XXS** | **38GB** | **48GB** | M4 Max 64GB | aggressive 3-bit, imatrix-calibrated |
| **Q3_K_M** | **48GB** | **56GB** | M4 Max 64GB | balanced 3-bit |
| **IQ4_XS** | **52GB** | **64GB** | M4 Max 96GB | imatrix 4-bit (best size/quality) |
| **Q4_K_M** | **60GB** | **72GB** | M4 Max 128GB | **recommended** for most users |
| **Q5_K_M** | **72GB** | **88GB** | M2 Ultra 192GB | higher fidelity |
| **Q6_K** | **85GB** | **96GB** | M2 Ultra 192GB+ | near-original quality |

> Multimodal users: also download `mmproj-*-BF16.gguf` (or `mmproj-*-Q6_K.gguf`) and use with `llama-server --mmproj` or `llama-mtmd-cli`.

## Hardware Reality Check

| Your System | IQ3 (38GB) | Q4 (60GB) | Q5 (72GB) | Q6 (85GB) |
|-------------|:----------:|:---------:|:---------:|:---------:|
| Mac mini M4 16GB | ❌ | ❌ | ❌ | ❌ |
| Mac mini M4 Pro 24GB | ❌ | ❌ | ❌ | ❌ |
| Mac M4 Max 64GB | ✅ tight | ❌ | ❌ | ❌ |
| **Mac M4 Max 96GB** | ✅ | ✅ tight | ❌ | ❌ |
| **Mac M4 Max 128GB** | ✅ | ✅ | ✅ tight | ❌ |
| **Mac M2 Ultra 192GB** | ✅ | ✅ | ✅ | ✅ |
| **Mac M3 Ultra 512GB** | ✅ comfortable | ✅ | ✅ | ✅ |

## How to run

### Ollama (text-only)

```bash
ollama pull batiai/llama4-scout:q4
ollama run batiai/llama4-scout:q4
```

### llama.cpp (text + vision via mmproj)

```bash
# Download GGUF + mmproj
hf download batiai/Llama-4-Scout-17B-16E-Instruct-GGUF \
    --include "*Q4_K_M*" --include "mmproj-*-Q6_K.gguf" \
    --local-dir ./llama4-scout

# Run with vision
llama-mtmd-cli \
    -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    --image input.jpg -p "Describe this image."

# Or as a server
llama-server -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    -ngl 99 -c 32768 --port 8080
```

## Model details

- **Source**: [meta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct)
- **Architecture**: `Llama4ForConditionalGeneration` — 109B total / 17B active MoE
- **Experts**: 16 routed (top-1 per token) — efficient sparse MoE
- **Multimodal**: text backbone + vision encoder (mmproj 분리)
- **Original precision**: BF16
- **License**: Meta Llama 4 Community License (commercial use OK for most, see link)

## BatiAI signing

All GGUFs in this repo carry:
- `general.author = BatiAI`
- `general.url = https://flow.bati.ai`

## Why BatiAI?

- Quantized **directly** from official Meta BF16 weights — no re-quantization
- IQ + K-quant variants share the same wikitext-2-raw imatrix recipe as every BatiAI model
- Multimodal mmproj packaged together for one-stop multimodal usage
- Verified on Apple Silicon (M4 Max / M2 Ultra)

## License

Inherits Meta Llama 4 Community License. Commercial-friendly for organizations with < 700M MAU. See:
- [Llama 4 License](https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE)
- [Acceptable Use Policy](https://www.llama.com/llama4/use-policy)

## About BatiFlow

[BatiFlow](https://flow.bati.ai) — free on-device AI automation for Mac.

<!-- BENCH-START -->
*Benchmarks coming once Mac M4 Max / M2 Ultra measurements complete.*
<!-- BENCH-END -->