RL-MPQ LLaVA-1.5-13B VLM Evaluation
Collection
RL-MPQ fake-quantized LLaVA-1.5-13B vision-language models and their VLMEvalKit benchmark results (MMMU, MMBench, ScienceQA). • 7 items • Updated
How to use AvoCahDoe/qwen2-vl-7b-rlmpq-balanced with Transformers:
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AvoCahDoe/qwen2-vl-7b-rlmpq-balanced", device_map="auto")RL-MPQ fake-quantized VLM evaluation artifacts.
| Repo | AvoCahDoe/qwen2-vl-7b-rlmpq-balanced |
| Scenario | Balanced |
| Avg bits | 3.39 |
| Collection | RL-MPQ VLM — Qwen2-VL-7B |
| Full results | dataset |
| Model | avg_bits | group | MMMU | MMBench | ScienceQA | Avg | ΔMMMU | ΔMMBench | ΔScienceQA | ΔAvg |
|---|---|---|---|---|---|---|---|---|---|---|
| FP16 (baseline) | 16.0 | qwen2vl_balanced | 49.67 | 78.1 | 85.08 | 70.95 | 0.0 | 0.0 | 0.0 | 0.0 |
| INT4 (bnb NF4) | 4.0 | qwen2vl_balanced | 39.33 | 21.13 | 75.31 | 45.26 | -10.34 | -56.97 | -9.77 | -25.69 |
| RL-MPQ Balanced | 3.39 | qwen2vl_balanced | 46.67 | 75.0 | 84.23 | 68.63 | -3.0 | -3.1 | -0.85 | -2.32 |
artifacts/figures/ — plots for the model groupartifacts/benchmark_table.csv — FP16 / INT4 / RL-MPQ accuracieseval_results.json — structured eval metadata (if present)