kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF

Muse-Glimmer-30B — ROCmFP4 for AMD Strix Halo (gfx1151)

✅ most quant variants of any public build — 4 ftypes in one repo (next: 3)

Verified against Hugging Face repository metadata for all 2 public ROCmFP4 builds of this base model. Size and file facts only — no third-party build was benchmarked.

Four ROCmFP4 quantisations of Meta's Muse-Glimmer-30B built for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, bundled with the DFlash speculative drafter and vision projector. ROCmFP4 is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp; Muse Glimmer support exists only in current upstream — this build ports the model forward into the ROCmFPX base so the two can meet.

Metric	Result
Quantization	ROCmFP4 (ggml types 100–106), 4 variants
Model size	13.80 – 16.87 GiB
Effective BPW	4.25 – 4.50 (see table)
Tested hardware	AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified
GPU	Radeon 8060S, gfx1151
ROCm version	7.2.4
32K decode (fastest variant)	20.31 tok/s
Prompt processing	not separately instrumented — see Not yet measured
DFlash decode	4.13 tokens accepted per target pass, n_max=15
Peak memory	19.99 GiB resident (model + drafter + projector + 32K KV)
Context validated	32768 only — see Not yet measured
Tool calling	6/7 on a 7-case suite (parity with upstream)
Reasoning	yes — reasoning_content / content split
Vision	yes — verified on spatial ground truth, requires -fa off

Why this build?

• 20.31 tok/s on the FAST variant vs 16.65 tok/s for Meta's fastest official GGUF (kquant-17gb) — same box, same flags, same drafter: 1.22×
• 13.80 GiB vs Meta's 15.61 GiB — 1.8 GiB smaller and faster
• Four ftypes published so you can pick the size/speed/verbosity point you want
• Text, vision and DFlash speculative decoding all work from one download
• Chat-format parser ported, so no to=self<|message|> control tokens leak into output
• Every published file validated before upload; nothing shipped unverified

Which file should I use?

Ryzen AI Max+ 395, ROCm 7.2.4, DFlash drafter at --spec-draft-n-max 15, -fa on, ctx 32768, batch 1, temperature 0. Warm medians of 9 generations; the first call after load is discarded.

Build	ftype	Size	BPW	TG 32K	Quality
ROCmFP4-FAST	103	13.80 GiB	4.25	20.31	3/3
ROCmFP4-STRIX_LEAN	106	14.00 GiB	4.38	18.72	3/3
ROCmFP4-STRIX	105	14.17 GiB	4.36	17.24	3/3
ROCmFP4-BASE	100	16.87 GiB	4.50	15.90	3/3
Meta kquant-17gb (reference)	—	15.61 GiB	—	16.65	3/3
Meta kquant-17gb on Vulkan	—	15.61 GiB	—	6.10	3/3

Start with FAST. BASE is both the slowest and the largest — it is published for completeness, not because anyone should choose it.

⚠ The faster files write shorter answers

Some of the speed comes from terser output, not only from faster decode. Median words per answer on identical prompts:

Build	Median words
STRIX (105)	377
STRIX_LEAN (106)	306
BASE (100)	301
FAST (103)	259

The quality check is substring-based and cannot distinguish "more concise" from "less thorough." If answer depth matters more than throughput, prefer STRIX. This is a real trade, not a free win.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF --local-dir muse

llama-server \
  -m muse/muse-glimmer-30B-ROCmFP4-FAST.gguf \
  --mmproj muse/mmproj-kquant.gguf \
  --spec-type draft-dflash \
  --model-draft muse/dflash-kquant.gguf \
  --spec-draft-n-max 15 \
  --chat-template-kwargs '{"reasoning_strength":"medium"}' \
  -ngl 999 -fa off -dio --jinja \
  -c 32768 --host 127.0.0.1 --port 8080

Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106). Stock llama.cpp rejects these tensor types.

Three flags that matter more than which file you pick

Flag	Why
--chat-template-kwargs '{"reasoning_strength":"..."}'	Template defaults to high. At high this model spent an entire 1200-token budget deliberating on a real refactor task and returned no visible answer at all. medium answered in 38.1s, low in 32.6s.
-fa on (text) / -fa off (vision)	-fa off costs +21% at ctx 32768 but is mandatory for images. Run separate endpoints if you serve both.
--spec-draft-n-max 15	DFlash block size is 16; one slot holds the previously accepted token.

⛔ --reasoning-budget does not work on this model. Values 256 and -1 produced byte-identical runs at temperature 0 — the flag is not enforced on peg-native format. Use reasoning_strength instead.

Verified hardware

Hardware	GPU	ROCm	Status	TG 32K	Notes
Ryzen AI Max+ 395 (Strix Halo)	Radeon 8060S / gfx1151	7.2.4	✅ Tested by KingJones	20.31	128 GB unified
Any Vulkan backend	—	—	❌ Known incompatible	—	rejects ggml type 101 at parse time
gfx1201 / RDNA4	—	—	❓ Untested	—
NVIDIA / CUDA	—	—	❌ Known incompatible	—	ROCmFP4 is a ROCm-only tensor format

Vulkan is impossible, not merely slow. The backend rejects these files at parse time:

gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 101. should be in [0, 43)

Vulkan's type table ends at 43; ROCmFP4 types are 100–106. No flag changes this. For reference, Meta's k-quant does run on Vulkan and measured 6.10 tok/s versus 16.65 on ROCm on this box, so Vulkan is not a useful path for this model in any case.

Speculative decoding (DFlash)

Muse Glimmer has no MTP tensors — zero in both the base checkpoint and the quants. It speculates using DFlash against a separate 5-layer drafter, which is what Meta's own recipe prescribes.

Metric	Value
Setting	--spec-type draft-dflash --spec-draft-n-max 15
Tokens accepted per target pass	4.13 (median, range 2.85–6.65)
Per-token acceptance	~21%
Drafter memory	1.52 GiB

⚠️ Per-token acceptance is a misleading statistic for a block drafter. DFlash proposes 15 tokens in one forward pass; ~21% acceptance means ~4.13 tokens land per target pass, which is healthy. Judge block drafters on tokens-per-pass.

n-gram speculation is not a substitute here. Measured on the same build: ngram-map-k reached 42.9% acceptance — double DFlash's — yet ran 45% slower on code-transform work (15.20 vs 27.49 tok/s), because it proposes far fewer tokens per pass.

A ROCmFP4 drafter is included but is not the default. It measured 1.008× against Meta's k-quant drafter on a quiet box — inside noise, acceptance unchanged. The drafter is ~1.5 GiB of a ~17 GiB working set, so shrinking it 8.5% moves total memory traffic by well under 1%. Shipped because it is valid, not because it is faster.

Tool calling

7-case suite, run against this build and against Meta's k-quant on the upstream binary as a reference:

Case	This build	Upstream reference
multi-arg (string/int/bool)	✅	✅
nested object argument	✅	✅
enum constraint	✅	✅
correctly declines (no spurious call)	✅	✅
multi-turn tool-result follow-up	✅	✅
streaming tool call	✅	✅
two parallel calls in one turn	❌	❌
Total	6/7	6/7

The parallel-call failure is the model's, not the quantisation's — Meta's own weights on upstream's own parser fail identically. Sequential agent loops are unaffected.

Raw example:

{"name": "book_flight",
 "arguments": {"passenger": {"name": "Alice Smith", "age": 34},
               "route": "LHR-JFK", "cabin": "business"}}

Agentic loop

A multi-step loop (list → move → observe → finish) over a directory of loose files, 3 runs at temperature 0.7: 3/3 completed the task correctly, 0 cases of claiming an action without emitting a tool call.

Vision

Works, and is verified for spatial correctness rather than plausible-sounding output: a four-quadrant colour image is scored on whether each colour lands in the right corner. A misapplied attention mask names colours confidently but places them wrongly, so this test distinguishes a working port from a fluent-but-broken one. 3/3.

Requires -fa off — ggml_flash_attn_ext aborts on Muse's per-layer sparse-window masks.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (upstream tree; only it has the muse-glimmer converter)
python convert_hf_to_gguf.py <MODEL_DIR> --outtype bf16 --outfile muse-glimmer-30B-BF16.gguf

# 2. quantize with the ROCmFPX build (only it has ggml types 100-106)
llama-quantize muse-glimmer-30B-BF16.gguf muse-glimmer-30B-ROCmFP4-FAST.gguf 103

Source: meta-models/Muse-Glimmer-30B BF16 safetensors, 1436 tensors → 55.7 GB BF16 GGUF (731 text tensors) → ROCmFP4.

The model was ported forward into the ROCmFPX base in three stages:

1. Text graph, arch registration and converter. Three API gaps bridged: is_swa_impl → swa_layers, n_layer() from method to field, and the NVFP4-only output-scale argument (null on the ROCmFP4 path).
2. Vision tower — required teaching the older base's build_vit to accept per-layer attention masks at all; it previously took no mask parameter. Added as an overload so the ~32 other vision models calling it are untouched.
3. Chat-format parser, so harmony-style channel output is parsed rather than leaking to=self<|message|> into content.

Files

Three distinct networks, not parts of one — llama.cpp loads them via --model, --model-draft and --mmproj. There is no merged single-file format.

File	Size	Role
muse-glimmer-30B-ROCmFP4-FAST.gguf	13.80 GiB	model — fastest, recommended
muse-glimmer-30B-ROCmFP4-STRIX_LEAN.gguf	14.00 GiB	model
muse-glimmer-30B-ROCmFP4-STRIX.gguf	14.17 GiB	model — most verbose output
muse-glimmer-30B-ROCmFP4-BASE.gguf	16.87 GiB	model — not recommended
dflash-kquant.gguf	1.52 GiB	DFlash drafter (Meta's, unmodified) — use this
dflash-ROCmFP4-STRIX.gguf	1.39 GiB	our ROCmFP4 drafter — works, 1.008× (a wash)
mmproj-kquant.gguf	1.30 GiB	vision projector (Meta's, unmodified)

Not yet measured

Listed explicitly so nobody mistakes absence for a pass. These are genuine gaps, not claims:

Test	Status
Context scaling (2K / 8K / 16K / 64K / 128K)	❓ only 32768 measured
Prompt-processing tok/s, isolated	❓ not separately instrumented
Sustained generation (1K / 4K tokens)	❓ not measured
Perplexity / KL divergence vs BF16	❓ not measured
MMLU-Pro, GPQA, GSM8K, HumanEval+, MBPP+	❓ not run
Long-context needle retrieval	❓ not run
DFlash n-max sweep (2 / 4 / 8 / 24)	❓ only n=15 measured
5-run statistics with std dev	⚠️ 9 samples per arm, median reported; std dev not published
Independent reproduction	❓ none yet

Measurement conditions: the tok/s figures were taken on a machine that also served other traffic during the run. The ordering across builds is wide enough to be reliable; the exact ratios are not trustworthy to three significant figures. A re-run on a quiesced box is planned.

Quality caveat: the 3/3 figure is a smoke check over factual recall, arithmetic and instruction-following, scored by substring match. It is a regression guard against a broken quantisation, not a benchmark suite, and it does not measure answer depth. No claim of "no quality loss" is made — that would require the perplexity and standardized evaluations listed above.

Independent results

None yet. If you run this build, please open a discussion with: hardware, GPU, OS, ROCm version, runtime commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent reproductions will be listed separately from author benchmarks and carry more weight.

Known issues

1. Vulkan/CUDA/CPU cannot load these files — ROCmFP4 is a ROCm-only tensor format.
2. Vision requires -fa off, costing ~21% on text at 32K context.
3. Parallel tool calls fail — model-level, reproduced identically on Meta's own weights.
4. Small max_tokens returns empty content — the budget goes to reasoning_content. Allow several hundred tokens.
5. --reasoning-budget is not enforced on this model; use reasoning_strength.

License and attribution

Base model, DFlash drafter and vision projector are Meta's, under the base model's licence. ROCmFP4 quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains the quantised weights and the measurements above.

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Repository	Largest model file	Variant	Ships	Downloads	Likes
RadixArk/Muse-Glimmer-NVFP4	4.00 GiB	NVFP4	safetensors	40	5
kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF (this repo)	16.87 GiB	STRIX	4 model files, drafter, vision	0	1
Preyazz/Muse-Glimmer-30B-NVFP4	17.37 GiB	NVFP4	safetensors	0	5
cloudnathan5/Muse-Glimmer-30B-NVFP4	18.63 GiB	NVFP4	safetensors	0	2
RedHatAI/Muse-Glimmer-30B-NVFP4	18.63 GiB	NVFP4	safetensors	0	7
vmlinux/Muse-Glimmer-30B-ROCmFPX-GGUF	26.77 GiB	ROCmFPX	3 model files, drafter, vision	0	13

Base model: meta-models/Muse-Glimmer-30B. Generated from Hub metadata; download counts move over time.

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.