Instructions to use moonshotai/Kimi-VL-A3B-Thinking with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use moonshotai/Kimi-VL-A3B-Thinking with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-VL-A3B-Thinking", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("moonshotai/Kimi-VL-A3B-Thinking", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use moonshotai/Kimi-VL-A3B-Thinking with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "moonshotai/Kimi-VL-A3B-Thinking"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-VL-A3B-Thinking",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/moonshotai/Kimi-VL-A3B-Thinking

SGLang

How to use moonshotai/Kimi-VL-A3B-Thinking with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "moonshotai/Kimi-VL-A3B-Thinking" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-VL-A3B-Thinking",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "moonshotai/Kimi-VL-A3B-Thinking" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "moonshotai/Kimi-VL-A3B-Thinking",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use moonshotai/Kimi-VL-A3B-Thinking with Docker Model Runner:
```
docker model run hf.co/moonshotai/Kimi-VL-A3B-Thinking
```

Kimi-VL-A3B-Thinking

Commit History

Transformers v5 support (#19)

7d99e22
verified

courage17340

hmellor HF Staff commited on Jan 30

remove _validate_images_text_input_order as well (#18)

481598c
verified

zhouzaida commited on Aug 18, 2025

Update README.md

3702c31
verified

teowu commited on Jun 27, 2025

Update README.md

33ec794
verified

teowu commited on Jun 21, 2025

Update README.md

acec364
verified

teowu commited on Jun 18, 2025

update generation config (#15)

16681d8
verified

zhouzaida commited on Apr 20, 2025

set do_sample as true (#14)

ceb7ae6
verified

zhouzaida commited on Apr 20, 2025

add generation_config.json (#13)

c95c453
verified

zhouzaida commited on Apr 20, 2025

remove log from tokenizer (#12)

5b6731b
verified

zhouzaida commited on Apr 20, 2025

support training (#10)

e784cd9
verified

zhouzaida commited on Apr 14, 2025

tokenizer can decode tensor for vllm test (#9)

622aeb0
verified

zhouzaida commited on Apr 14, 2025

fix GenerationMixin ImportError in transformers main branch (#6)

3f0f648
verified

zhouzaida commited on Apr 13, 2025

Update README.md

50260a0
verified

Nealeon commited on Apr 12, 2025

Update README.md

bfd4bc4
verified

Nealeon commited on Apr 12, 2025

Update README.md

edb5925
verified

Nealeon commited on Apr 12, 2025

Add library_name metadata and link to paper (#3)

17bce8a
verified

Nealeon

nielsr HF Staff commited on Apr 12, 2025

can set attn_implementation (#4)

e957990
verified

zhouzaida commited on Apr 11, 2025

Update README.md

54d622b
verified

teowu commited on Apr 11, 2025

support infer in cpu device (#2)

3ce416b
verified

zhouzaida commited on Apr 11, 2025

upload

f321730

zhouzaida commited on Apr 9, 2025

initial commit

c8504ea
unverified

zhouzaida commited on Apr 9, 2025

Commit History

Transformers v5 support (#19) 7d99e22 verified

remove _validate_images_text_input_order as well (#18) 481598c verified

Update README.md 3702c31 verified

Update README.md 33ec794 verified

Update README.md acec364 verified

update generation config (#15) 16681d8 verified

set do_sample as true (#14) ceb7ae6 verified

add generation_config.json (#13) c95c453 verified

remove log from tokenizer (#12) 5b6731b verified

support training (#10) e784cd9 verified

tokenizer can decode tensor for vllm test (#9) 622aeb0 verified

fix GenerationMixin ImportError in transformers main branch (#6) 3f0f648 verified

Update README.md 50260a0 verified

Update README.md bfd4bc4 verified

Update README.md edb5925 verified

Add library_name metadata and link to paper (#3) 17bce8a verified

can set attn_implementation (#4) e957990 verified

Update README.md 54d622b verified

support infer in cpu device (#2) 3ce416b verified

upload f321730

initial commit c8504ea unverified

Transformers v5 support (#19)

7d99e22
verified

remove _validate_images_text_input_order as well (#18)

481598c
verified

Update README.md

3702c31
verified

Update README.md

33ec794
verified

Update README.md

acec364
verified

update generation config (#15)

16681d8
verified

set do_sample as true (#14)

ceb7ae6
verified

add generation_config.json (#13)

c95c453
verified

remove log from tokenizer (#12)

5b6731b
verified

support training (#10)

e784cd9
verified

tokenizer can decode tensor for vllm test (#9)

622aeb0
verified

fix GenerationMixin ImportError in transformers main branch (#6)

3f0f648
verified

Update README.md

50260a0
verified

Update README.md

bfd4bc4
verified

Update README.md

edb5925
verified

Add library_name metadata and link to paper (#3)

17bce8a
verified

can set attn_implementation (#4)

e957990
verified

Update README.md

54d622b
verified

support infer in cpu device (#2)

3ce416b
verified

upload

f321730

initial commit

c8504ea
unverified