---
license: other
license_name: glm-5.3
license_link: LICENSE
base_model: zai-org/GLM-5.3
base_model_relation: quantized
tags:
- mlx
- apple-silicon
- glm_moe_dsa
- mixture-of-experts
- 4-bit
pipeline_tag: text-generation
library_name: mlx
---

# GLM-5.3-MLX-4bit

MLX (Apple Silicon) build of [**GLM-5.3**](https://huggingface.co/zai-org/GLM-5.3) — 744B-parameter
`glm_moe_dsa` MoE (256 experts, top-8; MLA with DeepSeek-V3.2-style sparse attention) — quantized
to **4-bit**.

**These files are modified**: converted from the upstream **bfloat16** release
([GLM-5.3-BF16](https://huggingface.co/zai-org/GLM-5.3-BF16)) to MLX and quantized; the architecture
is unchanged. The multi-token-prediction layer (78) is not included.

## Runtime — read this

This checkpoint bundles `glm_moe_dsa.py` (declared via `model_file`) and needs it:

```bash
pip install -U mlx-lm
mlx_lm.generate --model pipenetwork/GLM-5.3-MLX-4bit --trust-remote-code --prompt "..." --max-tokens 300
```

mlx-lm's own `glm_moe_dsa` builds a lightning indexer on all 78 layers, but GLM-5.2/5.3 ship
indexer weights on 21 (`indexer_types`: the other 57 "shared" layers reuse the previous full layer's
top-k selection). A strict load of the release fails with 285 missing parameters; `mlx_lm.load`
loads leniently and leaves those 57 indexers at random initialisation. Prompts up to 2048 tokens are
unaffected (the indexer is bypassed below `index_topk`); beyond that, 57 layers attend to keys
chosen by random projections. The bundled runtime implements the schedule as the reference does,
plus the reference's fp32 indexer scores and router logits and the indexer LayerNorm epsilon.
Tiny-config parity against `transformers` 5.16 is **4e-7** with the sparse path live, cached decode
exact; strict loading of this checkpoint reports zero missing and zero unexpected tensors. Details and
tests: [https://github.com/PipeNetwork/glm53-mlx](https://github.com/PipeNetwork/glm53-mlx).

## Size and what is quantized

**418.6 GB** on disk. RAM: 512 GB, tight.

| group | share of parameters | this build |
|---|---:|---|
| routed experts (`switch_mlp`, 75 layers × 256) | 724.8B (97.5%) | 4-bit, group 64 |
| attention (MLA), shared experts, dense layers 0–2, embeddings, `lm_head` | 18.4B (2.5%) | 4-bit, group 64 |
| lightning indexer (21 layers), MoE router + correction bias, norms | 0.3B | as stored (bf16 / fp32) |

Source precision: the FP8 release is a lossy derivative of the bf16 one (dequantized FP8 weights
differ from bf16 by up to 1.6e-2 on values of 0.46 — half an e4m3 step). The ladder row `fp8` is the FP8 release itself measured against bf16: its error is the floor any FP8-sourced build inherits.

## Quality

Two measurements, because at 744B most of the ladder cannot be loaded on a 512 GB machine:

**Per-layer divergence vs bf16** (`scripts/eval_ladder.py`): every decoder layer run in bf16 and in
each recipe on identical inputs (16,384 tokens of wikitext-2), *teacher-forced* (each layer
sees bf16 inputs — isolates its own damage) and *free-running* (each recipe feeds itself — what
inference does). Relative L2 error of the layer output; lower is better.

| recipe | teacher-forced (mean over layers) | free-running (final layer) | cosine (final) |
|---|---:|---:|---:|
| 8bit | 0.00685 | 0.13119 | 0.98945 |
| 6bit | 0.01465 | 0.16736 | 0.98389 |
| 5bit | 0.02651 | 0.22521 | 0.97272 |
| 4bit | 0.05161 | 0.35740 | 0.93390 |
| mixed-4_8bit | 0.02524 | 0.24951 | 0.96710 |
| mixed-3_6bit | 0.05242 | 0.42380 | 0.90624 |
| fp8 | 0.01741 | 0.17321 | 0.98320 |

**Perplexity** on wikitext-2 (test), 288,627 tokens in 141 windows of 2048, for the builds
that fit this machine, scored on identical windows:

| build | size | perplexity [95% CI] |
|---|---:|---|
| [4bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-4bit) | 418.6 GB | 2.8636 [2.6681, 3.0714] |
| [mixed-4_8bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-mixed-4_8bit) | 427.8 GB | 2.7420 [2.5533, 2.9477] |
| [mixed-3_6bit](https://huggingface.co/pipenetwork/GLM-5.3-MLX-mixed-3_6bit) | 332.6 GB | 3.0338 [2.8366, 3.2386] |

**How to read this.** The ladder is the only measurement that covers the whole set: 8-bit is closest to bfloat16 (free-running error 0.131, cosine 0.989 after 78 layers), 6-bit next (0.167). The upstream **FP8 release scores 0.173 — between 6-bit and 5-bit** — so any build converted from FP8 starts below our 6-bit, which is why these are converted from bf16. 5-bit (0.225) and mixed 4/8-bit (0.250) are close, and only the mixed build fits a 512 GB Mac; uniform 4-bit (0.357) is the tight 512 GB option; **mixed 3/6-bit (0.424) loses to uniform 4-bit** — it leads for the first ten layers, then 3-bit expert damage compounds — and is published for the 384 GB fit, not for quality. Perplexity on the builds that fit this machine is below.

Greedy generation (a collapse detector, not a ranking) is coherent on every published build.

## License

[GLM-5.3 license](LICENSE), as the upstream model. Port code: [https://github.com/PipeNetwork/glm53-mlx](https://github.com/PipeNetwork/glm53-mlx).