Instructions to use meta-models/Muse-Glimmer-30B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meta-models/Muse-Glimmer-30B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="meta-models/Muse-Glimmer-30B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("meta-models/Muse-Glimmer-30B")
model = AutoModelForMultimodalLM.from_pretrained("meta-models/Muse-Glimmer-30B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use meta-models/Muse-Glimmer-30B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meta-models/Muse-Glimmer-30B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-models/Muse-Glimmer-30B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/meta-models/Muse-Glimmer-30B

SGLang

How to use meta-models/Muse-Glimmer-30B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meta-models/Muse-Glimmer-30B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-models/Muse-Glimmer-30B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meta-models/Muse-Glimmer-30B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-models/Muse-Glimmer-30B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use meta-models/Muse-Glimmer-30B with Docker Model Runner:
```
docker model run hf.co/meta-models/Muse-Glimmer-30B
```

Request for HF Team / Community. ZeroGPU Space for testing the model.

#27

by NodeLinker - opened 1 day ago

Discussion

NodeLinker

1 day ago

•

edited 1 day ago

Glimmer has 2 ZeroGPU Spaces, but I can't test it in either of them. So far, the model isn't available via any API, and I don't have a GPU that can handle Glimmer. Kaggle / Google Colab are out of the question — the model is too large.

There is a Space for Gemma 4 12B:
https://huggingface.co/spaces/huggingface-projects/gemma-4-12b-it

Could someone create a similar Space, but with Glimmer instead of Gemma 4 12B, ideally with mmproj, DFlash and https://huggingface.co/meta-models/Muse-Glimmer-30B/discussions/21#6a7a177bd1648bc345eda3d1 ?

ZeroGPU is currently available for free to everyone, even without a Pro subscription. Just saying, in case someone doesn't know and that's holding them back.

I'll keep trying to make a Space myself, by the way. The two Spaces that exist right now — it's not that they don't work, they're VibeCoded (nothing wrong with that per se), but people keep changing them all the time, and they lack flexible settings for usage. Why reinvent the wheel when there's a good skeleton right here?
https://huggingface.co/spaces/huggingface-projects/gemma-4-12b-it

Please vote with a reaction if you support this initiative.

NodeLinker

1 day ago

•

edited 1 day ago

UPD: Since the template is written for Transformers, rewriting it for llama.cpp would be more difficult, and Transformers is a more native environment for Hugging Face.

Here is the commit where model support was added:
https://github.com/huggingface/transformers/commit/fe95f5423d65951cf63055d519dd7fa5ae12eb8d

And here is DFlash:
https://huggingface.co/meta-models/Muse-Glimmer-30B-assistant

The commit already includes DFlash support for Transformers.

So the original idea of doing this via llama.cpp was a bad one. It needs to be done with Transformers.

NodeLinker

1 day ago

•

edited about 24 hours ago

Long story short, I've been struggling with this. I still haven't managed to get it running and generate even a single token. What really gets on my nerves is that HF always takes more duration than what you specify, so test runs just to check if the model works end up eating your limit to the point where you can only make 2 test requests on a single account. I've got 5 accounts here, and I'm tired of switching between them and messing around (Actually, I used two accounts for this. I've already tried it on OpenRouter, and I can't be bothered switching to another account to play LLM casino just to run something I don't even need anymore. Plus, the build takes way longer than getting a response from the LLM—which will be 100% wrong anyway—and then I'd have to go through this whole cycle again). And what's more, it's clear that the GPU is sitting idle, but HF thinks it's under heavy load right now and needs to spend duration on its usage.

I don't know how much time you have to waste to get ZeroGPU running on a Free account, but I'm sick of it. And yes, I tried running it without DFlash.

The model is now available on OpenRouter—thanks to Together. Just a huge thank you to Together and all the contributors to vLLM.

But working with HF ZeroGPU is hell). Sure, it's cool that it's now available for free, but what's the point if it still doesn't work?

Although it's probably me messing up, and I just need to move it to the standard chat template — use the loading code and Gemma logic provided by Hugging Face itself, but change it to Glimmer — and everything will be hunky-dory.

NodeLinker

about 20 hours ago

So in the end, I managed to do it after how long? 6 hours?

I just decided I had to finish it, went to my third account. Opened the HuggingFace documentation and saw that you simply can't access it with less than 48 GB of VRAM. Downloaded the unsloth quant and it worked!!! Then I added DFlash and it worked too, even though I thought it was a lie. If the response had fewer than 16 tokens, the model would loop. The EOT token would come out, but it kept going due to the DFlash predictions. Well, I fixed that and it seems to be all good. I also added the ability to adjust the minimum number of seconds the GPU is reserved for. No one does that, but it's a very useful feature.

Again, this is all AI Slop, so there might be bugs that only God knows about, but thank you HuggingFace for the ZeroGPU Space, and I hope you'll forgive me for using three accounts for this "project").

When I was about to give up completely, I finally decided to finish the task, and I'm so glad everything I wanted actually worked out.

https://huggingface.co/spaces/NodeLinker/Muse-Glimmer-30B-DFlash

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment