Instructions to use ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModel

processor = AutoProcessor.from_pretrained("ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview", trust_remote_code=True)
model = AutoModel.from_pretrained("ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview

SGLang

How to use ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview with Docker Model Runner:
```
docker model run hf.co/ISTA-DASLab/Kimi-K2.5-P48-NVFP4-W4A4-Preview
```

kwanhee commited on May 23

Commit

36ec5d8

verified ·

1 Parent(s): 8d8bf1b

Add files using upload-large-folder tool

Browse files

Files changed (29) hide show

model-00011-of-00062.safetensors +3 -0
model-00020-of-00062.safetensors +3 -0
model-00021-of-00062.safetensors +3 -0
model-00023-of-00062.safetensors +3 -0
model-00024-of-00062.safetensors +3 -0
model-00025-of-00062.safetensors +3 -0
model-00027-of-00062.safetensors +3 -0
model-00028-of-00062.safetensors +3 -0
model-00029-of-00062.safetensors +3 -0
model-00030-of-00062.safetensors +3 -0
model-00031-of-00062.safetensors +3 -0
model-00032-of-00062.safetensors +3 -0
model-00033-of-00062.safetensors +3 -0
model-00034-of-00062.safetensors +3 -0
model-00035-of-00062.safetensors +3 -0
model-00036-of-00062.safetensors +3 -0
model-00037-of-00062.safetensors +3 -0
model-00038-of-00062.safetensors +3 -0
model-00039-of-00062.safetensors +3 -0
model-00041-of-00062.safetensors +3 -0
model-00042-of-00062.safetensors +3 -0
model-00043-of-00062.safetensors +3 -0
model-00044-of-00062.safetensors +3 -0
model-00045-of-00062.safetensors +3 -0
model-00047-of-00062.safetensors +3 -0
model-00049-of-00062.safetensors +3 -0
model-00051-of-00062.safetensors +3 -0
model.safetensors.index.json +3 -0
tiktoken.model +3 -0

model-00011-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:52747bf8932075cba052f421da1ff4322e4e4920f6fc13c35bd9eb6c35bc08e2
+size 9809193080

model-00020-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7816f899ee18bca74d1d774673977c38461e108f4327417133c0daebd118a102
+size 9809197704

model-00021-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:559e566f127df08a7847c5506f431c7596c35c8b4d6bc8f1bccf51615bf0709e
+size 9809197704

model-00023-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c307001aafa91f1c17f000823f3565e1813fd556c0da72f93d7c2a2784286822
+size 9809197704

model-00024-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a9197a3633639036f4aabea7b4b2080a943f191d3dda19661faf1485087f4c55
+size 9809197704

model-00025-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfcc7f64e82825426ffc3a1fc666b1bfc4cb341cd332bdc878e611b86a2ee1c0
+size 9809197704

model-00027-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c0b29852de9de6c1e8d8165362cfbeb3aad107b9afe169ecc1675c7f0f319fc0
+size 9809197704

model-00028-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:34cdb93f08dd6d60be954774e8e7880bdc2982f4912a40896ee3171090e811b8
+size 9809197704

model-00029-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:44948fc23ceb5790ea097a65f8256b88f9abf38cbc323d6045331277013f5552
+size 9809197704

model-00030-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da3fb1e872e5b22a1c24099f2fb90d0f9ffcc7e6f42ac7c496c6a590b25d7cd8
+size 9809197704

model-00031-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dcf65a2c335c71036b19fdd1d8feedd32030265aad3ef3e961a75303d843b6ec
+size 9809197704

model-00032-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b5a8b7493e2211112c815426497ba68f59e39cbb79917f4ef6740f4e10d1309
+size 9809197704

model-00033-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:abb8228dfd3926e2d10ed0b807d7e9acccf37af350de3ebc52ea9a3ce69ff0af
+size 9809197704

model-00034-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7758d5790a14cf1917f7b0d63c8aef7db6ef3d7e232f9aedc76eb61f6c3decc
+size 9809197704

model-00035-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:22b845b10f1bf2297964f2bcac0471eb2398e25678d4e2763c9de630df09719a
+size 9809197704

model-00036-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f22b53bfa9d59ea6d08b116929592cef779a14c29b87ae99b4725a37ec483996
+size 9809197704

model-00037-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:860bc54c423a6c51cc555656aa84e7a605de371dac233229b781cde55d6e4ede
+size 9809197704

model-00038-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:25fdf9b8b922ad3a61278683cb0860004880ccb1ac25de0d09e12f8145fd5d9e
+size 9809197704

model-00039-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1b0be32c6251ac0e4a9be1270ea830704920679b184d98c46c62b2ae1bb70658
+size 9809197704

model-00041-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:30c874729cfe6d51b3096dd8bbfa30c18cbc3aaa14a8896fb4484dd539a09365
+size 9809197704

model-00042-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e073125ba20f1ffbba53602ec00c1dd23f2043de41f071481e47aa3297069cb9
+size 9809197704

model-00043-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f45b22972122a165785e41cb5cf8efa875eb109572d5a8de2576554f77267180
+size 9809197704

model-00044-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a834aeae38d07fe17e90166ffab9281aaf6f76b708430ab20fc0963d8115cc3e
+size 9809197704

model-00045-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f984251057be59fa73c774bbee25920cf14843b1cebe5c4ca362ecaeb6b6c05
+size 9809197704

model-00047-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:53cf525486087547227bb4af7247ff6f5e4a61f6cc31849fd79ae4a8afba4c11
+size 9809197704

model-00049-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:77f2aa70336ef596a17cb06211f96b258be736e580104c367d00a4da14468530
+size 9809197704

model-00051-of-00062.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9915b7d66054b448697779b31b94e7a4ca3f09467b42274e6f3aa1e2954e311c
+size 9809197704

model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7c2cd4c12fc7475189b97ef25fea5bc7373ad632afb2a119deea8a67f7d75220
+size 39756072

tiktoken.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b6c497a7469b33ced9c38afb1ad6e47f03f5e5dc05f15930799210ec050c5103
+size 2795286