Instructions to use migtissera/Tess-4-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use migtissera/Tess-4-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="migtissera/Tess-4-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("migtissera/Tess-4-27B")
model = AutoModelForMultimodalLM.from_pretrained("migtissera/Tess-4-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use migtissera/Tess-4-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "migtissera/Tess-4-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "migtissera/Tess-4-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/migtissera/Tess-4-27B

SGLang

How to use migtissera/Tess-4-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "migtissera/Tess-4-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "migtissera/Tess-4-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "migtissera/Tess-4-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "migtissera/Tess-4-27B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use migtissera/Tess-4-27B with Docker Model Runner:
```
docker model run hf.co/migtissera/Tess-4-27B
```

Tess in a merged model

#15

by nightmedia - opened Jul 20

Discussion

nightmedia

Jul 20

Thank you for a great model!

I am working on a complex model assembly, and included Tess in my last merge:

https://huggingface.co/nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-Tess-1M-qx64-hi-mlx

Full metrics are on the model card: although not intended for coding and more for RP, this has shown to work really well even in 1M context, easily explained by Tess' long context training.

-G

migtissera

Owner Jul 20

Nice! Glad you're having fun with it :)

nightmedia

Jul 20

•

edited Jul 20

The 9B is also excellent, shows very high metrics by itself. I used it as a narrative device in a RP merge

migtissera/Tess-4-9B
          arc   arc/e boolq hswag obkqa piqa  wino
qx86-hi   0.611,0.809,0.897,0.751,0.446,0.789,0.726

Qwen3.5-9B-Holodeck-Tess-Bradbury-Orwell
qx86-hi   0.655,0.833,0.893,0.721,0.472,0.778,0.713

Qwen3.5-9B-Instruct
qx86-hi   0.553,0.712,0.892,0.689,0.436,0.765,0.691

https://huggingface.co/nightmedia/Qwen3.5-9B-Holodeck-Tess-Bradbury-Orwell-q8-hi-mlx

nightmedia

Jul 20

While I'm here, I have a question:

I've been trying to download the 35B model source, but I get stuck at the last safetensor, always exactly at this position: 5.21G/6.06G

Was this an incomplete upload, or is just HF being weird?

migtissera

Owner Jul 20

Oh wow, it improved Qwen3.5-9B-Instruct in all axes. Crazy! I didn't even benchmark it lol. I used the Qwen3.5-9B-Base and fine-tuned it with the Tess-4 dataset.

nightmedia

Jul 20

•

edited Jul 20

If you consider doing a granite-4.1-8B, I'll gladly merge it :)

That would be the western equivalent of the Qwen 9B, and I got some really good results on it. We need more people do do fun stuff with Granite, their infrastructure is easier to train than Qwen.

And yes, the dataset quality matters :)

migtissera

Owner Jul 20

Interesting! It's a model from IBM?

I nowadays use Tinker for fine-tuning, and this model is not supported in it. Do you guys use any of these models? https://tinker-docs.thinkingmachines.ai/tinker/models/

nightmedia

Jul 20

•

edited Jul 20

Yes, IBM released a series with 3B, 8B, 31B, and a 26B MoE. All great models, good RoPE handling and recall, fairly well trained. DavidAU had to add the HEAD to some of them because IBM forgot to put that in, and since it sort of works--they delivered it. People on HF added the missing parts. Great model.

Here is my latest on the 8B that has the same traces as the 9B

https://huggingface.co/nightmedia/granite-4.1-8B-Brainstone-PKD-TNG-Coder-Heretic-qx86-hi-mlx

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.527,0.730,0.861,0.697,0.450,0.792,0.687
q8-hi    0.551,0.721,0.878,0.694,0.434,0.781,0.680
qx86-hi  0.555,0.720,0.877,0.695,0.442,0.781,0.684

granite-4.1-8b

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.486,0.666,0.875,0.636,0.450,0.766,0.631

The important bit is to train on a Heretic base

treadon/granite-4.1-8b-Abliterated-AND-Disinhibited

quant    arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.496,0.692,0.864,0.666,0.466,0.770,0.632
q8-hi    0.477,0.658,0.865,0.657,0.460,0.755,0.604

It unlocks a lot of good things. There is plenty of room at the top

Never used Tinker, first time I see it too :)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment