Instructions to use QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4")
model = AutoModelForMultimodalLM.from_pretrained("QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4

SGLang

How to use QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 with Docker Model Runner:
```
docker model run hf.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4
```

Muse Glimmer 30B — native NVFP4 W4A4 for Blackwell, 21.8 GiB

Blackwell W4A4 build of our QUASAR-trained Muse Glimmer 30B: 20% lower KL to BF16 than Red Hat's NVFP4 W4A4 checkpoint and 45% lower than RTN, with 93.2% top-1 agreement. All 416 decoder projections use NVFP4. Standard vLLM / compressed-tensors, no custom kernels.

NVFP4 / vLLM · W4A4 / Blackwell (this repo) · GGUF / llama.cpp · Muse Glimmer collection · QUASAR paper

Model	Precision	Size	KL ↓	top-1 ↑
QUASAR W4A4 (this model)	NVFP4 W4A4, 416 / 416	21.8 GiB	0.0530	93.2%
Red Hat NVFP4 (LLM Compressor PTQ)	NVFP4 W4A4, 416 / 416	21.8 GiB	0.0664	92.4%
RTN NVFP4 (no training)	NVFP4 W4A4, 416 / 416	21.8 GiB	0.0958	90.8%

With the same 4-bit activations but BF16 weights, KL is 0.0434; quantizing the weights with QUASAR adds only 0.0096 KL. KL / top-1: per-token forward KL(BF16 ‖ model) and top-1 agreement on 948 held-out agentic prompts of the BF16 model's own responses (1.19M response tokens), under a vLLM-verbatim simulation of the W4A4 NVFP4 activation path (per-token group-16 E2M1, dynamic FP8 local scales, static per-layer global scale) — same harness and prompts as the W4A16 card. Raw results in eval/.

Run it

pip install "vllm>=0.28"

vllm serve QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4 \
  --tool-call-parser muse_glimmer --reasoning-parser muse_glimmer --enable-auto-tool-choice \
  --max-model-len 131072

compressed-tensors nvfp4-pack-quantized with NVFP4 input_activations. On Blackwell (B200 / GB200 / RTX 5090), vLLM uses the native NVFP4 W4A4 path. For Hopper, use the dedicated W4A16 build.

Capability benchmarks: for GPQA-Diamond, MMLU-Pro, AIME'25, coding, tool use and RULER through 128K on the underlying QUASAR-trained weights, see the W4A16 reference build.

Technical details

All 416 decoder projections use NVFP4 E2M1 weights with group size 16 and FP8-E4M3 group scales. Embeddings, LM head, norms and vision tower remain BF16. This repo uses the same QUASAR-trained weights as the W4A16 checkpoint and adds calibrated NVFP4 activation scales for Blackwell W4A4 execution.

W4A16 / Hopper (the reference build): QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4
llama.cpp / Ollama (native Q4_0 QAT): QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF
Collection: Muse Glimmer 30B — QUASAR 4-bit QAT · Base model: meta-models/Muse-Glimmer-30B · Paper: arXiv:2608.13966
Running this checkpoint? Share benchmark results, deployment notes, or integrations in Discussions.

Citation

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}