MobileWorld-TEXT-8B

MobileWorld-TEXT-8B is a mobile GUI world model. Given the current screenshot and a candidate action, it predicts the next screen state as a complete text description.

Use this model when the downstream component needs a full textual description of the predicted next page rather than HTML code or only the changed region.

Input

The model is used as a multimodal chat model. Provide:

• One current GUI screenshot.
• A system prompt describing the full next-page prediction task.
• A user prompt containing the action description, action target, and relative coordinates.

System Prompt

You are a graphical user interface (GUI) agent. You are given an action, an action target, the relative coordinates of the object being manipulated, and a screenshot. Your task is to predict the state of the next page after performing this action. Note that you need to provide a complete and detailed description of the next page, not an incremental description based on changes to the current page.

User Prompt Template

<image> Predict the next page state via text description from this current screenshot using action description "{action_description}" and action tartget "{action_target}" and relative coordinates "{relative_coordinates}".

The word tartget is intentionally kept as in the training/inference template.

Fields

• action_description: natural-language action, for example click, scroll down, or input text: hello.
• action_target: target UI element or target point, for example point(536, 1280) or search input field.
• relative_coordinates: normalized coordinates in (x, y) format, for example (0.496296, 0.533333). Use N/A for actions without a coordinate target.

Output

The expected output is a complete natural-language description of the predicted next screen. It should describe the resulting page state, not just the delta from the current screen.

Example OpenAI-Compatible Call

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

def image_url(path):
    payload = base64.b64encode(open(path, "rb").read()).decode("utf-8")
    return f"data:image/png;base64,{payload}"

system_prompt = """You are a graphical user interface (GUI) agent. You are given an action, an action target, the relative coordinates of the object being manipulated, and a screenshot. Your task is to predict the state of the next page after performing this action. Note that you need to provide a complete and detailed description of the next page, not an incremental description based on changes to the current page."""

user_prompt = '<image> Predict the next page state via text description from this current screenshot using action description "input text: pizza" and action tartget "search input field" and relative coordinates "N/A".'

response = client.chat.completions.create(
    model="xwk123/MobileWorld-TEXT-8B",
    messages=[
        {"role": "system", "content": system_prompt},
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": image_url("screenshot.png")}},
                {"type": "text", "text": user_prompt},
            ],
        },
    ],
    max_tokens=4000,
    temperature=0.0,
)

next_page_description = response.choices[0].message.content.strip()