---
library_name: gguf
base_model: ByteDance/EvoQuality
language: en
license: apache-2.0
tags:
  - gguf
  - llama.cpp
  - image-quality-assessment
  - no-reference-iqa
  - perceptual-quality
  - aigc-quality
  - qwen2.5-vl
  - vision-language
  - quantized
pipeline_tag: image-text-to-text
---

# EvoQuality-IQA-GGUF

GGUF quants of [ByteDance/EvoQuality](https://huggingface.co/ByteDance/EvoQuality) for fast local NR-IQA scoring with llama.cpp. Six bench-validated variants (BF16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS) plus the shared mmproj-f16 vision sidecar.

EvoQuality is an 8B Qwen2.5-VL-7B-based no-reference image quality assessment model. It scores a single image on a 1-5 perceptual scale (1 = severely degraded, 5 = clean, sharp, artifact-free) and supports pairwise A-vs-B comparison. Upstream reports PLCC +25% and SRCC +27% over the Qwen2.5-VL-7B baseline.

This is the first community GGUF release for EvoQuality. The companion repo with conversion scripts, the bench harness, and the Docker harness is at [DoradusResearch/EvoQuality-IQA-GGUF](https://github.com/DoradusResearch/EvoQuality-IQA-GGUF). Methodology and use-case context are in the [release blog post](https://doradusresearch.ai/blog/evoquality-iqa-gguf-release/).

## Quant lineup

Bench: 99 images from the [AGIQA-3K](https://huggingface.co/datasets/strawhat/agiqa-3k) test split, stratified across the full MOS range. Each variant served via llama-server with the shared mmproj-f16 sidecar. `temperature=0`. Per-image scores in [bench-*.jsonl](./bench-Q5_K_M.jsonl), aggregated in [benchmarks-summary.json](./benchmarks-summary.json).

![Quant quality vs ground truth](./assets/quant-metrics-bar.png)

| Variant | File size | PLCC vs MOS | SRCC vs MOS | Δ PLCC vs BF16 | Recommendation |
|---|---|---|---|---|---|
| BF16 | 14.19 GB | 0.8033 | 0.7177 | reference | Highest fidelity |
| **Q8_0** | 7.54 GB | 0.8037 | 0.7183 | +0.05% | Use if ≥12 GB VRAM. Within bench noise of BF16. |
| Q6_K | 5.82 GB | 0.7924 | 0.7124 | -1.35% | Q5_K_M is a better Pareto point. |
| **Q5_K_M** | 5.07 GB | 0.7999 | 0.7158 | -0.43% | **Production sweet spot.** Best quality-per-byte. |
| **Q4_K_M** | 4.36 GB | 0.7940 | 0.7138 | -1.16% | Compact. Fits ~8 GB VRAM. |
| IQ4_XS | 3.96 GB | 0.7738 | 0.7029 | -3.68% | Smallest. Measurable but bounded loss. |

All six variants exceed the Qwen2.5-VL-7B baseline (~0.615 PLCC) that EvoQuality is trained to lift.

### Per-variant scatter

![Per-variant scatter grid](./assets/quant-comparison-grid.png)

### Overlay

![Overlay](./assets/quant-comparison-overlay.png)

## VRAM guide

| Your VRAM | Recommended | Resident (weights + mmproj + KV @ 8K × 4) |
|---|---|---|
| ≥ 16 GB | Q8_0 | ~10-12 GB |
| 12 GB | Q5_K_M | ~8 GB |
| 8 GB | Q4_K_M | ~6.5 GB |
| 6 GB | IQ4_XS | ~6 GB (tight) |
| Reference | BF16 | ~18 GB |

All quants share the same mmproj-f16 vision encoder (1.35 GB). Vision-tower quantization had a substantially larger quality impact than language-tower quantization in our internal tests; mmproj is kept at f16 across the spread.

## Quickstart — llama-server

```bash
llama-server \
  --model    evoquality-iqa-Q5_K_M.gguf \
  --mmproj   mmproj-evoquality-iqa-f16.gguf \
  --host 0.0.0.0 --port 8259 \
  --ctx-size 8192 \
  --n-gpu-layers 999 \
  --parallel 4 --cont-batching \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --alias evoquality-iqa
```

Score an image:

```bash
curl -s http://localhost:8259/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "evoquality-iqa",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
        {"type": "text", "text": "Rate the perceptual quality of this image on a scale from 1 to 5, where 1=severely degraded (heavy noise, compression artifacts, blur, broken content) and 5=high quality (clean, sharp, artifact-free). Respond with a single integer 1-5 followed by a brief justification."}
      ]
    }],
    "max_tokens": 64,
    "temperature": 0
  }'
```

Response:

```json
{"choices":[{"message":{"content":"3\nThe image is moderately sharp but has visible compression artifacts and slight aliasing on diagonal edges."}}]}
```

## Python

```python
from huggingface_hub import hf_hub_download
gguf  = hf_hub_download("Doradus-AI/EvoQuality-IQA-GGUF", "evoquality-iqa-Q5_K_M.gguf")
mmp   = hf_hub_download("Doradus-AI/EvoQuality-IQA-GGUF", "mmproj-evoquality-iqa-f16.gguf")
```

## Reproducing the build

Full pipeline in [DoradusResearch/EvoQuality-IQA-GGUF](https://github.com/DoradusResearch/EvoQuality-IQA-GGUF). Conversion is vanilla llama.cpp:

```bash
huggingface-cli download ByteDance/EvoQuality --local-dir ./hf
python3 llama.cpp/convert_hf_to_gguf.py ./hf --outfile ./evoquality-iqa-BF16.gguf --outtype bf16
python3 llama.cpp/convert_hf_to_gguf.py ./hf --outfile ./evoquality-iqa-Q8_0.gguf --outtype q8_0
python3 llama.cpp/convert_hf_to_gguf.py ./hf --mmproj --outfile ./mmproj-evoquality-iqa-f16.gguf --outtype f16
for Q in Q6_K Q5_K_M Q4_K_M IQ4_XS; do
  llama.cpp/build/bin/llama-quantize ./evoquality-iqa-BF16.gguf ./evoquality-iqa-$Q.gguf $Q 8
done
```

llama.cpp commit verified working: `b9010-d05fe1d7d` (June 2026). Older revisions can mishandle Qwen2.5-VL mmproj.

## Bench methodology

- **Dataset**: [strawhat/agiqa-3k](https://huggingface.co/datasets/strawhat/agiqa-3k) test split (598 images, `mos_quality` labels).
- **Sample**: 100 images, evenly spaced across the full MOS range (0.00 → 4.37). Effective n = 99.
- **Prompt**: identical 1-5 rating template across every variant. `temperature=0`, `max_tokens=64`. Three retries with exponential backoff.
- **Metrics**: PLCC = `scipy.stats.pearsonr(MOS, predicted_score)`. SRCC = `scipy.stats.spearmanr(MOS, predicted_score)` (tie-corrected ranks).

Per-image score data ships in this repo: [bench-BF16.jsonl](./bench-BF16.jsonl) and the five quant variants. Aggregated summary at [benchmarks-summary.json](./benchmarks-summary.json).

## License

Apache-2.0, inherited from upstream [ByteDance/EvoQuality](https://huggingface.co/ByteDance/EvoQuality).

## Citation

```bibtex
@misc{bytedance2025evoquality,
  title  = {EvoQuality: Self-Evolving Vision-Language Model for No-Reference Image Quality Assessment},
  author = {ByteDance},
  year   = {2025},
  url    = {https://huggingface.co/ByteDance/EvoQuality}
}
@misc{agiqa3k,
  title  = {AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment},
  author = {Li, Chunyi et al.},
  year   = {2023},
  url    = {https://huggingface.co/papers/2306.04717}
}
```