---
license: apache-2.0
base_model: Qwen/Qwen2.5-VL-3B-Instruct
tags:
  - quantized
  - gptq
  - vision-language-model
  - vlm
---

# Qwen__Qwen2.5-VL-3B-Instruct__gptq_int4

This is a **GPTQ** (4-bit) quantized version of [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct).

## Quantization Details

- **Method**: GPTQ
- **Bits**: 4
- **Base model**: Qwen/Qwen2.5-VL-3B-Instruct
- **Group size**: 128
- **Quantized portion**: LLM backbone only
- **Vision tower**: FP16 (unchanged)
- **Loading**: AutoModelForImageTextToText / AutoModelForCausalLM (trust_remote_code where applicable)

## Usage

```python
from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "{REPO_ID}",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("{REPO_ID}", trust_remote_code=True)
```

Replace `{REPO_ID}` with the repo ID of this model.

## Original Model

See [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) for the original FP16 model.