Instructions to use zzzrw/GEM-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zzzrw/GEM-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="zzzrw/GEM-2B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForSeq2SeqLM

processor = AutoProcessor.from_pretrained("zzzrw/GEM-2B")
model = AutoModelForSeq2SeqLM.from_pretrained("zzzrw/GEM-2B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zzzrw/GEM-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zzzrw/GEM-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zzzrw/GEM-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/zzzrw/GEM-2B

SGLang

How to use zzzrw/GEM-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zzzrw/GEM-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zzzrw/GEM-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zzzrw/GEM-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zzzrw/GEM-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use zzzrw/GEM-2B with Docker Model Runner:
```
docker model run hf.co/zzzrw/GEM-2B
```

GEM-2B

File size: 3,455 Bytes

33819a5
 
6d02d87
 
 
 
 
 
 
 
2f4c9be
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8854e3d
2f4c9be
 
 
 
 
2adbafa
0e727ef
2adbafa

---
license: mit
datasets:
- zzzrw/GEM-250K
language:
- en
- zh
base_model:
- Qwen/Qwen3-VL-2B-Instruct
pipeline_tag: image-text-to-text
---

<div align="center">
<h1>GEM: Generative Supervision Helps Embodied Intelligence</h1>

<p align="center">
    <a href="https://zhaorw02.github.io/">Ruowen Zhao</a><sup>1</sup>,
    Bangguo Li<sup>1</sup>,
    <a href="https://liuzuyan.github.io/">Zuyan Liu</a><sup>1,2,†</sup>,
     Yinan Liang<sup>1</sup>,
    <a href="https://jamesyjl.github.io/">Junliang Ye</a><sup>1</sup>,
    <a href="https://liuff19.github.io/">Fangfu Liu</a><sup>1</sup>,
  <br>                             
   Diankun Wu<sup>1</sup>,
  <a href="https://thuwzy.github.io/">Zhengyi Wang</a><sup>1</sup>,
    <a href="https://yuxumin.github.io/">Xumin Yu</a><sup>2</sup>,
     <a href="https://raoyongming.github.io/">Yongming Rao</a><sup>2,✉</sup>,
   <a href="https://ancientmooner.github.io/">Han Hu</a><sup>2</sup>,
    <a href="https://ml.cs.tsinghua.edu.cn/~jun/index.shtml">Jun Zhu</a><sup>1,✉</sup>
  <br>
    <sup>†</sup>Project Lead.<sup>✉</sup>Corresponding Author.
  <br>
    <sup>1</sup>Tsinghua University,
    <sup>2</sup>Tencent Hunyuan
</p>
<a href='https://zhaorw02.github.io/GEM/'><img src="https://img.shields.io/badge/Project-Page-Green" alt="Project Page"></a>
<a href="https://arxiv.org/abs/2605.28548"><img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv" alt="Paper"></a>
<a href="https://github.com/zhaorw02/GEM/"><img src="https://img.shields.io/badge/GitHub-Repo-181717?logo=github&logoColor=white" alt="GitHub"></a>
<a href="https://huggingface.co/zzzrw/GEM-2B/"><img src="https://img.shields.io/badge/Models-HuggingFace-yellow?logo=huggingface" alt="Models"></a>
<a href="https://huggingface.co/datasets/zzzrw/GEM-250K/"><img src="https://img.shields.io/badge/Dataset-HuggingFace-yellow?logo=huggingface" alt="Dataset"></a>

</div>
<div align="center">
<video src="https://huggingface.co/datasets/zzzrw/GEM-250K/resolve/main/assets/GEM-demo.mp4" controls autoplay muted loop width="85%"></video>
</div>

Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within Vision-Language-Action frameworks. However, a significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments. In this paper, we introduce **GEM**, a Generative-supervised Embodied vision-language Model designed to bridge this divide. We propose integrating a depth map generation task directly into the VLM pre-training phase.   By training this generative objective jointly with the main model, we observe substantial improvements in embodied intelligence, significantly enhancing both semantic understanding and physical operation capabilities. To support this paradigm, we curate and release GEM-4M, a comprehensive large-scale dataset featuring a mixture of grounding, reasoning, and planning data paired with high-quality depth supervision. Extensive experiments demonstrate that GEM achieves state-of-the-art results across diverse embodied benchmarks. Furthermore, our deployed action model, GEM-VLA, exhibits vastly superior task execution abilities in both simulation environments and real-world evaluations.

<div align="center">
<img src="assets/overview.png" alt="GEM Teaser" width="85%">
</div>