Instructions to use darkc0de/Muse-Glimmer-30B-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use darkc0de/Muse-Glimmer-30B-heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="darkc0de/Muse-Glimmer-30B-heretic")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("darkc0de/Muse-Glimmer-30B-heretic")
model = AutoModelForMultimodalLM.from_pretrained("darkc0de/Muse-Glimmer-30B-heretic", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use darkc0de/Muse-Glimmer-30B-heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "darkc0de/Muse-Glimmer-30B-heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darkc0de/Muse-Glimmer-30B-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/darkc0de/Muse-Glimmer-30B-heretic

SGLang

How to use darkc0de/Muse-Glimmer-30B-heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "darkc0de/Muse-Glimmer-30B-heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darkc0de/Muse-Glimmer-30B-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "darkc0de/Muse-Glimmer-30B-heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "darkc0de/Muse-Glimmer-30B-heretic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use darkc0de/Muse-Glimmer-30B-heretic with Docker Model Runner:
```
docker model run hf.co/darkc0de/Muse-Glimmer-30B-heretic
```

GGUFs please? :)

by je0923 - opened 25 days ago

Discussion

je0923

25 days ago

Especially Q8? :D

darkc0de

Owner 25 days ago

•

edited 25 days ago

Just for your information
Any models you'd like high quality GGUFs for in pretty much every quant imaginable....
Go to:
https://huggingface.co/mradermacher/model_requests

Post the models link/address (not just the name) in a new discussion

darkc0de changed discussion status to closed 25 days ago

darkc0de changed discussion status to open 25 days ago

darkc0de

Owner 25 days ago

)

darkc0de

Owner 25 days ago

It's a fast free service to the community that produces the highest quality GGUFs usually in about a day.

That way you don't have to ask or wait anymore in the future

Just saying 👍

je0923

25 days ago

Wait that's free? How on earth do they afford to do that? O:

darkc0de

Owner 25 days ago

They are good people doing good things for the community just because

Hard to believe they actually still exist huh?

GeoMaciolek

23 days ago

Interestingly, Bartowski beat them this time!

https://huggingface.co/bartowski/darkc0de_Muse-Glimmer-30B-heretic-GGUF

darkc0de

Owner 23 days ago

Yep, Bartowski is another reputable source of high quality GGUFs

darkc0de

Owner 23 days ago

While we're on the topic, if your looking for "do it yourself right now" options

This space works on most models:
https://huggingface.co/spaces/ggml-org/gguf-my-repo

Also you can try Unsloth Studio's Export feature

jwkirkland

23 days ago

Thank you.

ArushCodes

22 days ago

Wow meta is building good model but forgot to actually make real muse spark a good model *it can't build a calculator bro 🫪... By the way You're Good man ! Appreciate for your models

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment