---
license: apache-2.0
pipeline_tag: image-text-to-text
library_name: transformers
base_model: Qwen/Qwen3-VL-8B-Instruct
tags:
  - reward-model
  - gui-agent
  - ui-genie
  - sft
  - discrete-preference
  - qwen3-vl
---

# Qwen3-VL-8B-Instruct-UI-Genie

An SFT fine-tuned reward model based on [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct), trained on the [UI-Genie-RM-517k](https://huggingface.co/datasets/UI-Genie/UI-Genie-RM-517k) dataset for GUI agent trajectory evaluation.

This model classifies GUI agent actions by generating a **discrete preference token**:
- `<|+|>` → action is **correct** (score = 1.0)
- `<|-|>` → action is **wrong** (score = 0.0)

A judge prompt is appended after the agent's tool-call response, and the model greedily decodes one classification token.

## Prompt Format

The model was trained on a multi-turn format:

```
[system]    You are a helpful assistant. + mobile_use tool spec (with screen resolution)
[user]      The user query: <goal>
            Task progress (You have done the following operation on the current device):
            Step1: <action>  <image>
            ...
            StepN: <action>; <image>
[assistant] <tool_call>{"name": "mobile_use", "arguments": {...}}</tool_call>
[user]      Was the agent's action above correct given the current screen state?
            Answer with exactly one token: <|+|> for correct, <|-|> for wrong.
[assistant] ← model generates <|+|> or <|-|>
```

## Intended Use

Use this model as a process reward model (PRM) to evaluate GUI agent actions on mobile UIs. It is well-suited for:

- Step-level correctness classification during agent rollouts
- Best-of-N action selection
- Filtering training data by action quality

## Inference

### Requirements

```bash
pip install vllm transformers torch pillow
```

### Scoring with vLLM (LoRA mode)

```python
import os
from vllm import LLM, SamplingParams
from vllm.lora.request import LoRARequest
from transformers import AutoProcessor
from PIL import Image

MODEL_PATH = "Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie"
BASE_MODEL  = "Qwen/Qwen3-VL-8B-Instruct"

POS_TOKEN = "<|+|>"
NEG_TOKEN = "<|-|>"

JUDGE_PROMPT = (
    "Was the agent's action above correct given the current screen state? "
    "Answer with exactly one token: <|+|> for correct, <|-|> for wrong."
)

SYSTEM_PROMPT = (
    "You are a helpful assistant.\n\n# Tools\n\n"
    "You may call one or more functions to assist with the user query.\n\n"
    "You are provided with function signatures within <tools></tools> XML tags:\n"
    "<tools>\n"
    '{"type": "function", "function": {"name": "mobile_use", '
    '"description": "Use a touchscreen to interact with a mobile device. '
    "The screen's resolution is 540x1200.\", "
    '"parameters": {"properties": {"action": {"type": "string", '
    '"enum": ["click", "long_press", "swipe", "type", "key", "system_button", "open", "wait", "terminate"]}, '
    '"coordinate": {"type": "array"}, "text": {"type": "string"}, "button": {"type": "string"}}, '
    '"required": ["action"]}}}\n</tools>\n\n'
    "For each function call, return a json object within <tool_call></tool_call> XML tags:\n"
    "<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>"
)

# Load model (LoRA on top of base)
llm = LLM(
    model=BASE_MODEL,
    dtype="bfloat16",
    enable_lora=True,
    max_lora_rank=64,
    gpu_memory_utilization=0.7,
    max_model_len=8192,
    limit_mm_per_prompt={"image": 10},
    enforce_eager=True,
)
processor = AutoProcessor.from_pretrained(BASE_MODEL, max_pixels=1_048_576)
tokenizer = processor.tokenizer

pos_id = tokenizer.encode(POS_TOKEN, add_special_tokens=False)[0]
neg_id = tokenizer.encode(NEG_TOKEN, add_special_tokens=False)[0]

lora_request = LoRARequest("rm_lora", 1, MODEL_PATH)
sampling_params = SamplingParams(max_tokens=1, temperature=0.0)


def score_action(goal, prior_steps_text, tool_call_response, screenshot):
    """
    Score a GUI agent action.

    Args:
        goal:                 Task goal string.
        prior_steps_text:     String like "Step1: tap search\nStep2: type query\n"
        tool_call_response:   The agent's raw tool_call response string to judge.
        screenshot:           PIL.Image of the current screen state.

    Returns:
        float: 1.0 (correct), 0.0 (wrong), or 0.5 (undecided).
    """
    user_content = [
        {"type": "text", "text": f"The user query: {goal}\nTask progress (...): {prior_steps_text}; "},
        {"type": "image"},
    ]

    messages = [
        {"role": "system",    "content": SYSTEM_PROMPT},
        {"role": "user",      "content": user_content},
        {"role": "assistant", "content": tool_call_response.strip()},
        {"role": "user",      "content": JUDGE_PROMPT},
    ]

    prefix_text = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

    output = llm.generate(
        [{"prompt": prefix_text, "multi_modal_data": {"image": [screenshot]}}],
        sampling_params,
        lora_request=lora_request,
    )[0]

    gen_id = output.outputs[0].token_ids[0] if output.outputs[0].token_ids else None
    if gen_id == pos_id:
        return 1.0
    if gen_id == neg_id:
        return 0.0
    return 0.5   # neither token — treated as undecided


# Example
screenshot = Image.open("screenshot.png").convert("RGB")
tool_call  = '<tool_call>\n{"name": "mobile_use", "arguments": {"action": "click", "coordinate": [540, 120]}}\n</tool_call>'

score = score_action(
    goal="Tap the search button",
    prior_steps_text="Step1: opened the app\n",
    tool_call_response=tool_call,
    screenshot=screenshot,
)
print(f"Score: {score}")   # 1.0 = correct, 0.0 = wrong
```

### Pairwise evaluation with `rm_eval`

```bash
python eval_rm.py \
    --rm_path Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie \
    --datasets ui-genie \
    --mode discrete \
    --uigenie_jsonl /path/to/reward_data_rm_pairs_last5.jsonl \
    --uigenie_images_dir /path/to/images \
    --output_dir results/
```

## Training Details

| Field | Value |
|---|---|
| Base model | Qwen/Qwen3-VL-8B-Instruct |
| Training method | SFT (LoRA) |
| Training data | UI-Genie-RM-517k (64k pairs training split) |
| Output | Discrete preference token: `<\|+\|>` / `<\|-\|>` |
| Scoring | Greedy-decode 1 token → 1.0 / 0.0 / 0.5 |

## Related Models

- **[Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring](https://huggingface.co/Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring)** — Bradley-Terry scalar reward head built on top of this model.

## Citation

```bibtex
@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
}
```