Instructions to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ahishamm/SmolDocling-256M-preview-mlx-fp16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ahishamm/SmolDocling-256M-preview-mlx-fp16")
model = AutoModelForMultimodalLM.from_pretrained("ahishamm/SmolDocling-256M-preview-mlx-fp16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("ahishamm/SmolDocling-256M-preview-mlx-fp16")
config = load_config("ahishamm/SmolDocling-256M-preview-mlx-fp16")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ahishamm/SmolDocling-256M-preview-mlx-fp16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ahishamm/SmolDocling-256M-preview-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ahishamm/SmolDocling-256M-preview-mlx-fp16

SGLang

How to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ahishamm/SmolDocling-256M-preview-mlx-fp16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ahishamm/SmolDocling-256M-preview-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ahishamm/SmolDocling-256M-preview-mlx-fp16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ahishamm/SmolDocling-256M-preview-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ahishamm/SmolDocling-256M-preview-mlx-fp16 with Docker Model Runner:
```
docker model run hf.co/ahishamm/SmolDocling-256M-preview-mlx-fp16
```

SmolDocling-256M-preview-mlx-fp16 / added_tokens.json

ahishamm

Upload added_tokens.json with huggingface_hub

d08d051 verified about 1 year ago

raw

history blame contribute delete

3.67 kB

	{
	"</caption>": 49192,
	"</chart>": 49248,
	"</checkbox_selected>": 49211,
	"</checkbox_unselected>": 49213,
	"</doctag>": 49230,
	"</footnote>": 49195,
	"</form>": 49215,
	"</formula>": 49197,
	"</group>": 49228,
	"</key_": 49243,
	"</key_value_region>": 49217,
	"</list_item>": 49199,
	"</ordered_list>": 49224,
	"</otsl>": 49209,
	"</page_footer>": 49201,
	"</page_header>": 49203,
	"</paragraph>": 49220,
	"</picture>": 49205,
	"</reference>": 49222,
	"</section_header_level_": 49207,
	"</smiles>": 49251,
	"</unordered_list>": 49226,
	"</value_": 49245,
	"<caption>": 49191,
	"<chart>": 49247,
	"<checkbox_selected>": 49210,
	"<checkbox_unselected>": 49212,
	"<ched>": 49239,
	"<doctag>": 49229,
	"<ecel>": 49234,
	"<end_of_utterance>": 49279,
	"<fake_token_around_image>": 49189,
	"<fcel>": 49233,
	"<footnote>": 49193,
	"<form>": 49214,
	"<formula>": 49196,
	"<global-img>": 49152,
	"<group>": 49227,
	"<image>": 49190,
	"<key_": 49242,
	"<key_value_region>": 49216,
	"<lcel>": 49235,
	"<link_": 49246,
	"<list_item>": 49198,
	"<loc_": 49218,
	"<nl>": 49238,
	"<ordered_list>": 49223,
	"<otsl>": 49208,
	"<page_": 49231,
	"<page_break>": 49249,
	"<page_footer>": 49200,
	"<page_header>": 49202,
	"<paragraph>": 49219,
	"<picture>": 49204,
	"<reference>": 49221,
	"<rhed>": 49240,
	"<row_1_col_1>": 49153,
	"<row_1_col_2>": 49154,
	"<row_1_col_3>": 49155,
	"<row_1_col_4>": 49156,
	"<row_1_col_5>": 49157,
	"<row_1_col_6>": 49158,
	"<row_2_col_1>": 49159,
	"<row_2_col_2>": 49160,
	"<row_2_col_3>": 49161,
	"<row_2_col_4>": 49162,
	"<row_2_col_5>": 49163,
	"<row_2_col_6>": 49164,
	"<row_3_col_1>": 49165,
	"<row_3_col_2>": 49166,
	"<row_3_col_3>": 49167,
	"<row_3_col_4>": 49168,
	"<row_3_col_5>": 49169,
	"<row_3_col_6>": 49170,
	"<row_4_col_1>": 49171,
	"<row_4_col_2>": 49172,
	"<row_4_col_3>": 49173,
	"<row_4_col_4>": 49174,
	"<row_4_col_5>": 49175,
	"<row_4_col_6>": 49176,
	"<row_5_col_1>": 49177,
	"<row_5_col_2>": 49178,
	"<row_5_col_3>": 49179,
	"<row_5_col_4>": 49180,
	"<row_5_col_5>": 49181,
	"<row_5_col_6>": 49182,
	"<row_6_col_1>": 49183,
	"<row_6_col_2>": 49184,
	"<row_6_col_3>": 49185,
	"<row_6_col_4>": 49186,
	"<row_6_col_5>": 49187,
	"<row_6_col_6>": 49188,
	"<section_header_level_": 49206,
	"<smiles>": 49250,
	"<text_break>": 49232,
	"<ucel>": 49236,
	"<unordered_list>": 49225,
	"<value_": 49244,
	"<xcel>": 49237,
	"<\|reserved_special_token_3\|>": 49194,
	"<\|reserved_special_token_50\|>": 49241,
	"<\|reserved_special_token_61\|>": 49252,
	"<\|reserved_special_token_62\|>": 49253,
	"<\|reserved_special_token_63\|>": 49254,
	"<\|reserved_special_token_64\|>": 49255,
	"<\|reserved_special_token_65\|>": 49256,
	"<\|reserved_special_token_66\|>": 49257,
	"<\|reserved_special_token_67\|>": 49258,
	"<\|reserved_special_token_68\|>": 49259,
	"<\|reserved_special_token_69\|>": 49260,
	"<\|reserved_special_token_70\|>": 49261,
	"<\|reserved_special_token_71\|>": 49262,
	"<\|reserved_special_token_72\|>": 49263,
	"<\|reserved_special_token_73\|>": 49264,
	"<\|reserved_special_token_74\|>": 49265,
	"<\|reserved_special_token_75\|>": 49266,
	"<\|reserved_special_token_76\|>": 49267,
	"<\|reserved_special_token_77\|>": 49268,
	"<\|reserved_special_token_78\|>": 49269,
	"<\|reserved_special_token_79\|>": 49270,
	"<\|reserved_special_token_80\|>": 49271,
	"<\|reserved_special_token_81\|>": 49272,
	"<\|reserved_special_token_82\|>": 49273,
	"<\|reserved_special_token_83\|>": 49274,
	"<\|reserved_special_token_84\|>": 49275,
	"<\|reserved_special_token_85\|>": 49276,
	"<\|reserved_special_token_86\|>": 49277,
	"<\|reserved_special_token_87\|>": 49278
	}