portal-vlm: qwen25vl-lora-lm-vision
Grounding LoRA (r16/a32) on all 412 sites: 252 LM + 160 vision-tower (fused qkv included). Native dialect; best checkpoint at step 372/1489.
Part of the portal-vlm release - an independent replication of Ramp Labs' PorTAL (portable task adapters via hypernet-generated LoRA) extended to vision-language models on GUI grounding.
| ScreenSpot-v2 overall | web split | |
|---|---|---|
| this artifact | 82.6% | 76.0% |
- Base model:
Qwen/Qwen2.5-VL-3B-Instruct@66285546d2b821cf421d4f5eb2576359d3770cd3 - Training config:
configs/qwen25vl_lora_lm_vision.yaml - Eval record:
results/qwen25vl_lora_lm_vision.json
Reproduce this row without training
git clone https://github.com/robbym-dev/portal-vlm && cd portal-vlm && uv sync
uv run python scripts/eval.py --config configs/qwen25vl_lora_lm_vision.yaml --adapter hf:manihani4/portal-vlm-qwen25vl-lora-lm-vision
Standard PEFT LoRA adapter - also loadable directly with peft.PeftModel.from_pretrained on the pinned base model.
- Downloads last month
- 13
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for manihani4/portal-vlm-qwen25vl-lora-lm-vision
Base model
Qwen/Qwen2.5-VL-3B-Instruct