---
license: mit
base_model: zai-org/GLM-5.2-FP8
base_model_relation: quantized
tags:
  - glm
  - mixture-of-experts
  - int4
  - apple-silicon
  - ssd-streaming
library_name: safetensors
---

# GLM-5.2 — int4 container for iliria (full 744B, SSD-streamed)

The **full** GLM-5.2 (744B total / 39B active) quantized to int4 with an int8 MTP head, packaged
as the container that [**iliria**](https://github.com/PhilipJohnBasile/iliria) serves directly.

iliria runs this model on a single 128 GB Apple Silicon machine by **streaming experts from the
SSD** rather than holding all 744B parameters in memory. This repo exists so you don't have to
run that conversion yourself — otherwise the on-ramp is downloading ~372 GB of FP8 weights and
converting them locally for hours.

## This is not the pruned build

If you came from
[GLM-5.2-Demolition-q4a4-soul-MLX](https://huggingface.co/philipjohnbasile/GLM-5.2-Demolition-q4a4-soul-MLX),
note these are different artifacts and **not interchangeable**:

| | this repo | the demolition build |
|---|---|---|
| model | full 744B, unpruned | expert-pruned + LoRA-healed |
| size | ~384 GB (int4) | ~98 GB |
| format | iliria container (`out-*.safetensors`) | MLX |
| runs on | iliria | MLX / mlx-lm |

## Requirements

- Apple Silicon **M-Max / M-Ultra class with 128 GB** unified memory
- **~384 GB free SSD** (the weights are streamed from disk, not loaded into RAM)
- [iliria](https://github.com/PhilipJohnBasile/iliria)

## Use

```bash
hf download philipjohnbasile/GLM-5.2-colibri-int4-with-int8-mtp --local-dir ./glm52-int4
# then, from iliria:
./illi serve --model ./glm52-int4 --ram 60
```

That exposes an OpenAI-compatible endpoint. See the iliria README for the engine's flags,
context/RAM tuning, and measured throughput — expect deep-tier speeds, not chat speeds: this is
a 744B model reading experts off an SSD for every token.

## Contents

164 `out-*.safetensors` shards (int4 experts + int8 MTP head) plus `config.json`,
`generation_config.json`, and the tokenizer. Nothing else — no runtime state, no caches.

## Provenance

Quantized from [`zai-org/GLM-5.2-FP8`](https://huggingface.co/zai-org/GLM-5.2-FP8) (MIT).
Quantization only — no pruning, no retraining, no distillation; the full expert set is intact.
Produced by iliria's `tools/convert_fp8_to_int4.py`. Released under MIT, matching the base model.