omnifish123/vqa-rlvr-grpo-2b-main-sft

grpo_2b_main_sft — GRPO RLVR adapter for Qwen3-VL-2B (SFT-then-RL, VQAv2+GQA)

Part of vqa-rlvr: post-training Qwen3-VL for visual question answering on a single RTX 4090 (QLoRA SFT + GRPO with verifiable rewards).

Results (full eval sets; VQAv2 / GQA / CLEVR / TextVQA)

• Short template: 79.8 / 62.3 / 99.8 / 79.6
• Reasoning template: 69.1 / 54.3 / 87.1 / 72.3

Metrics: official VQA accuracy (VQAv2/TextVQA), normalized EM (GQA/CLEVR); harness cross-checked against lmms-eval. Full tables, configs, and per-run JSONs: https://github.com/guangboyu/vqa-rlvr.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3-VL-2B-Instruct", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "omnifish123/vqa-rlvr-grpo-2b-main-sft").merge_and_unload()

This adapter applies on top of the base with the vqa-rlvr-sft-2b adapter merged in first (load base → apply sft adapter → merge_and_unload() → apply this adapter). Training config is in run_config.json in this repo.

Erratum (2026-07)

A defect in the training-time correctness reward meant VQAv2 examples in this adapter's RL mix contributed no correctness signal (single-gold leave-one-out degeneracy; the correctness gradient came entirely from GQA). Evaluation numbers are unaffected (evaluation uses 10-annotator VQAv2 data). The checkpoint's behavior is as published; interpretation of the reward-design ablation should account for the defect. Fixed in repo commit 766fe8c; see the repository erratum for the affected-runs table and re-validation results.