MobileWorld-Delta-TEXT-8B

MobileWorld-Delta-TEXT-8B is a mobile GUI world model. Given the current screenshot and an action, it predicts the changes that will happen on the next screen.

Unlike MobileWorld-TEXT-8B, this model is trained to produce an incremental description: it should focus on what changes after the action rather than describing the entire next page from scratch.

Input

The model is used as a multimodal chat model. Provide:

• One current GUI screenshot.
• A user prompt containing the action description.

No separate system prompt is required by the original inference template.

User Prompt Template

You are an intelligent GUI agent capable of understanding GUIs and actions on mobile devices. Given the current GUI screenshot and input action, describe the changes that will occur on the next screen after the action is performed.
 The action is {action}.

The line break before The action is ... is kept from the training/inference template.

Field

• action: natural-language action description, for example click, scroll down, input text: hello, or open app: Gmail.

If you have a structured action object, first convert it to a compact action description. The evaluation code used the following style:

click                    # click or long_press action
input text: <text>       # input_text action
scroll <direction>       # scroll action
open app: <app_name>     # open_app action

Output

The expected output is an incremental text description of the next-screen change. It should answer: what will be different after applying the action to the current screenshot?

Example OpenAI-Compatible Call

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def image_url(path):
    payload = base64.b64encode(open(path, "rb").read()).decode("utf-8")
    return f"data:image/png;base64,{payload}"

user_prompt = (
    "You are an intelligent GUI agent capable of understanding GUIs and actions on mobile devices. "
    "Given the current GUI screenshot and input action, describe the changes that will occur on the "
    "next screen after the action is performed.\n The action is scroll down."
)

response = client.chat.completions.create(
    model="xwk123/MobileWorld-Delta-TEXT-8B",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": image_url("screenshot.png")}},
                {"type": "text", "text": user_prompt},
            ],
        },
    ],
    max_tokens=2000,
    temperature=0.0,
)

delta_description = response.choices[0].message.content.strip()