base_model: google/gemma-4-12B
license: gemma
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- quantized
- gemma
- gemma4
- image-text-to-text
- apple-silicon
- turboquant
gemma-4-12B — TurboQuant MLX MXFP4
google/gemma-4-12B @ 023679ed352de9bb66cc873c9009ce3482585c08 quantized pack, published as majentik/gemma-4-12B-TurboQuant-MLX-MXFP4.
Method
MLX quantization via mlx-vlm 0.6.3 (MXFP4, group_size 32); vision + audio towers retained in BF16 (not quantized).
Release line
Released under the TurboQuant line. RotorQuant and TurboQuant are this project's release labels for this pack, not distinct quantization algorithms — both brand repos for a given tier carry byte-identical weights, produced once and published under two names. No brand-specific speedup is claimed or measured for either label.
Modality
This pack is image-text-to-text capable: the vision and audio towers ship in BF16 alongside the quantized text tower, so image (and audio) inputs are supported end to end via mlx-vlm.
Base-model note: this is the raw completion model (not instruction-tuned). It is not instruction-aligned for multimodal Q&A — expect free-form continuation behavior rather than chat-style image description, even though the vision/audio towers are present.
mlx-vlm suppress-tokens note: the base model's generation_config.json carries suppress_tokens for six multimodal placeholder token ids so that transformers generation is clean. mlx-vlm does not honor suppress_tokens automatically, so text-only generation with mlx-vlm can leak placeholder tokens (observed as a degenerate A<image|>A<image|>... loop in smoke testing) unless you suppress them yourself:
from mlx_vlm import load, generate
model, processor = load("majentik/gemma-4-12B-TurboQuant-MLX-MXFP4")
suppress = {255999: -1e9, 256000: -1e9, 258880: -1e9, 258881: -1e9, 258882: -1e9, 258883: -1e9}
output = generate(
model, processor, prompt,
max_tokens=256,
logit_bias=suppress,
)
License
Governed by the Gemma Terms of Use. See the upstream repo for the full license text.