---
base_model: Qwen/Qwen2.5-VL-3B-Instruct
language:
- multilingual
library_name: transformers
license: other
pipeline_tag: image-text-to-text
tags:
- quantized
- gptq
- compressed-tensors
- sima-ai
---

# Qwen2.5-VL-3B-Instruct-GPTQ

This is a local post-training quantized checkpoint of [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) for later LLiMa compilation and deployment on Sima.ai hardware. It must not be uploaded as part of the current VLM batch.

**Source revision:** Not captured in the flattened local source directory. An immutable upstream revision must be recovered and recorded before any release or redistribution.

**Source license metadata:** qwen-research. This checkpoint remains subject to the upstream license, intended-use guidance, and limitations.

## Quantization

| Component | Method | Weight format | Effective targets |
| --- | --- | --- | ---: |
| Decoder Linear layers and `lm_head` | GPTQ | symmetric INT4, group size 256, static act-order | 253 |
| Vision encoder Linear layers | GPTQ | symmetric INT8, per-channel, static act-order | 162 |
| Projector/architecture exceptions | none | source BF16 | Explicitly recorded in `recipe.yaml` |

The `lm_head` follows the default GPTQ INT4/G256 policy; no INT8 escalation was made. Calibration used `lmms-lab/flickr30k` `test[:512]`, deterministic dataset order, 512 image-text samples, sequence length 2048, batch size 1, no concatenation, and no padding to maximum length. The exact target names and effective settings are saved in `recipe.yaml`.

## Evaluation

Full MMStar used all 1,500 examples, VLMEvalKit commit `7055d3010c38ccb5dcae1bc9535ca19c7fe5d79f`, deterministic generation, and `exact_matching` without an API judge.

| Checkpoint | Overall accuracy |
| --- | ---: |
| Source | 54.1333% |
| Quantized | 53.5333% |
| Absolute change | -0.6000 percentage points |
| Relative change | -1.1084% |

Evaluation date: 2026-07-16. Slurm jobs: source 4641, quantized 4642. The aggregate evidence row is stored in `vlmeval_results/mmstar_results.csv`; raw predictions and status files remain under `vlmeval_results/mmstar_full/`.

Validation: MMStar inference passed (1,500/1,500).

Full matched WikiText-2 word perplexity (`EleutherAI/wikitext_document_level`, `wikitext-2-raw-v1`, no example limit; 2026-07-18): source 13.8655; this checkpoint 15.5220; absolute degradation +1.6565; relative degradation +11.95%. Raw JSON evidence: `perplexity_results/vlm_full_wikitext/qwen25_vl_3b_{source,gptq_w4g256_visiongptq_w8}_v3.json`.

## Reproduction

This directory contains the exact `quantize.py`, `recipe.yaml`, and `versions.txt` used for the artifact:

```bash
python quantize.py \
  --model-path /path/to/source-model \
  --output-dir /path/to/new-quantized-model
```

## Environment

```text
Python: 3.13.2
torch: 2.11.0+cu128
CUDA: 12.8
transformers: 5.10.1
llm-compressor: 0.12.0
auto-round: 0.13.0
compressed-tensors: 0.17.1
```

## Deployment

This is the pre-LLiMa quantized Hugging Face artifact. LLiMa compiler output must remain a separate deployment artifact unless its format and redistribution are approved. No Hugging Face upload is authorized for this batch.

## Limitations

Quantization quality can vary by language, visual domain, prompt format, context length, and downstream runtime. The MMStar result does not replace application-specific validation. Audio components, when present, and explicitly excluded projectors remain in source precision as documented in `recipe.yaml`.