Sim-to-Real Transfer for Web Agents
Collection
Tasks and adapters: Qwen2.5-VL-7B trained on a simulated Best Buy, the real one, or a mix (SFT then GRPO). • 7 items • Updated
Qwen2.5-VL-7B SFT+GRPO: mixed (sim + real)
Best GRPO checkpoint of the mixed condition (round 5). The strongest adapter in the study on both sites.
Part of the Halluminate blog post on sim-to-real transfer for web agents. Tasks: goodbuy-bestbuy-mirrored-tasks. Training logs: W&B report.
Results (pass@1, 320 tasks per site, matched criterion)
| Eval on | Goodbuy (sim) | Bestbuy (real) |
|---|---|---|
| This adapter | 39.1% | 23.4% |
Usage
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
model = PeftModel.from_pretrained(base, "victoria-kp/qwen25vl-7b-simtoreal-grpo-mixed")
Training
GRPO from the corresponding SFT adapter: 8 rollouts per task, 8 tasks per round, 10 rounds, binary matched-criterion reward from a deterministic page-state verifier, KL 0.03 to the frozen SFT adapter, clip 0.2, lr 5e-6, temperature 1.0. Trained on the mixed 128+128 split. This is the checkpoint with the best held-out success (round 5).
Related
- Downloads last month
- 21
Model tree for victoria-kp/qwen25vl-7b-simtoreal-grpo-mixed
Base model
Qwen/Qwen2.5-VL-7B-Instruct