File size: 3,566 Bytes
86ebe22 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
license: apache-2.0
base_model: meta-models/Muse-Glimmer-30B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
- mlx
- muse_glimmer
- image-text-to-text
- mxfp8
- apple-silicon
---
# Muse-Glimmer-30B — MLX MXFP8
MLX **MXFP8** (8-bit microscaling float) quantization of
[`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B),
a ~30B dense causal transformer with a ~1.8B perception encoder, built for
autonomous agentic tasks on consumer hardware. Runs on Apple Silicon via
[mlx-vlm](https://github.com/Blaizzy/mlx-vlm). Stays **image-text-to-text** —
the vision tower and projector are kept in bf16.
| | |
| :---- | :---- |
| **Precision** | MXFP8 (E4M3 + E8M0 shared scale, group size 32) |
| **Bits per weight** | 8.751 bpw |
| **On-disk size** | 32.6 GB |
| **Quantized** | language model (incl. `lm_head`) |
| **Kept in bf16** | vision tower + vision adapter/projection |
| **Recommended RAM** | **32 GB+** unified memory |
This is the **higher-fidelity** build, for **32 GB+** Macs. On a 24 GB machine
it exceeds RAM and pages to swap (usable only very slowly); use the
[MXFP4 build](https://huggingface.co/sahilchachra/Muse-Glimmer-30B-MXFP4)
(18.6 GB) there instead.
## Verification
Quantized with `mlx_lm.quantize_model` (mode `mxfp8`, group 32), keeping the
vision path in bf16. The MLX implementation correctly handles this
architecture's non-standard pieces (per-layer NoPE on the full-attention layers,
`final_logit_softcapping`, `qk_scale_factor`, `output_multiplier`, gated
attention, centered RMSNorm).
MXFP8 was validated against the **MXFP4 build, which passed 6/6 arithmetic
prompts end-to-end with correct answers and coherent reasoning**. On a fixed
8-prompt set (arithmetic + open-ended), MXFP8's next-token predictions were
captured and compared to MXFP4:
| Metric | MXFP8 vs MXFP4 |
| :---- | :---: |
| top-1 next-token agreement | **8/8** |
| logit cosine similarity | **0.998** (min 0.997) |
Since MXFP8 uses more bits than the behaviorally-verified MXFP4 and agrees with
it this closely, it is at least as faithful to the base model. (Full token-by-
token generation was not benchmarked here because 32.6 GB exceeds the 24 GB test
machine's RAM; on a 32 GB+ Mac it generates at normal speed.)
## Usage (mlx-vlm)
```bash
pip install -U mlx-vlm # needs >= 0.6.12 for the muse_glimmer architecture
```
```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("sahilchachra/Muse-Glimmer-30B-MXFP8")
config = model.config
messages = [{"role": "user", "content": "What is 84 * 3 / 2?"}]
prompt = apply_chat_template(processor, config, messages, add_generation_prompt=True)
text = generate(model, processor, prompt, max_tokens=256, verbose=True)
```
For image input, pass an image to `apply_chat_template` / `generate` per the
mlx-vlm docs — the vision path is preserved in bf16.
**Recommended sampling** (from the base model card): `temperature=1.0`,
`top_p=0.95`, `top_k=64`. Reasoning strength is set via the system prompt
(`Reasoning strength: low|medium|high|xhigh`).
## Notes & limitations
- Inherits all capabilities and limitations of the base model. See the
[original model card](https://huggingface.co/meta-models/Muse-Glimmer-30B) and
[usage policy](https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/main/USAGE_POLICY.md).
- Quantized by [@sahilchachra](https://huggingface.co/sahilchachra) with MLX.
Original model © Meta Superintelligence Lab, Apache 2.0.
|