Gemma-4-12B-Abliterated — MLX MXFP4
MLX MXFP4 quantisation of huihui-ai/Huihui-gemma-4-12B-it-abliterated.
Changes: weights quantised to MXFP4 from the BF16 source with
mlx_vlm.convert. No fine-tuning, no merging, no re-alignment.
Measured
Converted and measured on one machine — Apple M3 Ultra, 96 GB unified memory, macOS 27 — as part of a full ladder. Every rung in this family came from the same BF16 source with the same group size, so bit width is the only variable between them.
| Size on disk | 6.40 GB |
| Perplexity | 254.121 |
| Relative to best rung in family | 1.82× |
| Throughput (1 req / 8 concurrent) | 33.0 / 99.1 tok/s |
Perplexity measured on allenai/tulu-3-sft-mixture, 192 samples of 512 tokens,
seed 123 — identical for every rung.
Perplexity is only comparable within this family. Tokenizers differ between model families, so a number here should never be compared against a different base model's. The
×column above is the meaningful one.
Usage
pip install mlx-vlm
mlx_vlm.generate --model shoemoney/Gemma-4-12B-Abliterated-MLX-mxfp4 --prompt "Hello" --max-tokens 256
Load with mlx-vlm, not mlx-lm — this architecture is registered in mlx-vlm.
Provenance
mlx_vlm.convert --hf-path huihui-ai/Huihui-gemma-4-12B-it-abliterated \
--mlx-path Gemma-4-12B-Abliterated-mxfp4 -q --q-mode mxfp4
License
apache-2.0, inherited from the base model. Attribution above.
- Downloads last month
- 238
4-bit
Model tree for shoemoney/Gemma-4-12B-Abliterated-MLX-mxfp4
Base model
google/gemma-4-12B
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Gemma-4-12B-Abliterated-MLX-mxfp4 shoemoney/Gemma-4-12B-Abliterated-MLX-mxfp4