philipjohnbasile/GLM-5.2-colibri-int4-with-int8-mtp

GLM-5.2 — int4 container for iliria (full 744B, SSD-streamed)

Explore the model guide · All public work

Release at a glance

	This artifact
Purpose	The unpruned GLM 5.2 expert set in an int4 container with an int8 MTP head for the iliria engine.
Runtime	iliria streams experts from SSD. The documented setup is an Apple Silicon 128 GB machine with roughly 384 GB of weight storage.
Status	Recorded runtime; see the evidence and limits below.
Tensor download	383.94 GB (357.57 GiB) of root .safetensors files, including any root sidecars. This is a file-size total, not peak RAM.
Read first	The on-disk payload is not resident RAM. These files are not interchangeable with the pruned MLX build; consult iliria for runtime limits.

The full GLM-5.2 (744B total / 39B active) quantized to int4 with an int8 MTP head, packaged as the container that iliria serves directly.

iliria runs this model on a single 128 GB Apple Silicon machine by streaming experts from the SSD rather than holding all 744B parameters in memory. This repo exists so you don't have to run that conversion yourself — otherwise the on-ramp is downloading ~372 GB of FP8 weights and converting them locally for hours.

This is not the pruned build

If you came from GLM-5.2-Demolition-q4a4-soul-MLX, note these are different artifacts and not interchangeable:

	this repo	the demolition build
model	full 744B, unpruned	expert-pruned + LoRA-healed
root tensor size	~384 GB (int4)	~105 GB / 98 GiB
format	iliria container (out-*.safetensors)	MLX
runs on	iliria	MLX / mlx-lm

Requirements

• Apple Silicon M-Max / M-Ultra class with 128 GB unified memory
• ~384 GB free SSD (the weights are streamed from disk, not loaded into RAM)
• iliria

Use

hf download philipjohnbasile/GLM-5.2-colibri-int4-with-int8-mtp --local-dir ./glm52-int4
# then, from iliria:
./illi serve --model ./glm52-int4 --ram 60

That exposes an OpenAI-compatible endpoint. See the iliria README for the engine's flags, context/RAM tuning, and measured throughput — expect deep-tier speeds, not chat speeds: this is a 744B model reading experts off an SSD for every token.

Contents

164 out-*.safetensors shards (int4 experts + int8 MTP head) plus config.json, generation_config.json, and the tokenizer. Nothing else — no runtime state, no caches.

Provenance

Quantized from zai-org/GLM-5.2-FP8 (MIT). Quantization only — no pruning, no retraining, no distillation; the full expert set is intact. Produced by iliria's tools/convert_fp8_to_int4.py. Released under MIT, matching the base model.