Anakonkai/vietnamese-traffic-sign-vqa-checkpoints

Vietnamese Traffic Sign VQA — Checkpoints

Model checkpoints for the Deep Learning course final project.

Contents

Path	Description	Size
a1_lstm/best.pt	A1: CLIP + PhoBERT + Co-Attention + LSTM decoder	~2.4 GB
a2_transformer/best.pt	A2: CLIP + PhoBERT + Co-Attention + Transformer decoder	~2.5 GB
b2_sft_lora/	B2-SFT: Qwen2.5-VL-3B QLoRA adapter (best checkpoint)	~153 MB
b2_dpo_lora/	B2-DPO: Qwen2.5-VL-3B DPO-aligned adapter (500 pairs, balanced)	~153 MB
stable_diffusion_lora/	SD LoRA: Stable Diffusion v1.5 LoRA on traffic sign images (12k steps)	~6 MB

Results (Full Test Set)

Model	VQA Acc	BLEU-4	ROUGE-L	BERTScore	Latency
A1 LSTM	0.9484	0.9602	0.9584	0.9631	~11 ms
A2 Transformer	0.9377	0.9476	0.9486	0.9713	~13 ms
B1 Zero-shot	0.1962	0.0350	0.2899	0.4753	~167 ms
B2-SFT QLoRA	0.9379	0.9494	0.9508	0.9111	~484 ms

Human evaluation (100 natural questions): B2-DPO 73% > B2-SFT 70% > A1 61%

Dataset

See Anakonkai/vietnamese-traffic-sign-vqa

Code

GitHub: https://github.com/Anakonkai/SDA

Usage

A1 / A2

import torch
from models.model_a import VQAModelA

model = VQAModelA(decoder_type="lstm")  # or "transformer"
ckpt = torch.load("a1_lstm/best.pt", map_location="cpu")
model.load_state_dict(ckpt["model_state_dict"])

B2-SFT

python evaluate/evaluate.py \
  --model b2 --backend qwen25 --load-in-4bit \
  --checkpoint b2_sft_lora/ \
  --data data/processed/annotations

Stable Diffusion LoRA

from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
pipe.load_lora_weights("stable_diffusion_lora/")
image = pipe("a Vietnamese street intersection with traffic signs").images[0]