Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,64 @@
+---
+license: mit
+base_model: zai-org/GLM-5.2-FP8
+base_model_relation: quantized
+tags:
+  - glm
+  - mixture-of-experts
+  - int4
+  - apple-silicon
+  - ssd-streaming
+library_name: safetensors
+---
+
+# GLM-5.2 — int4 container for iliria (full 744B, SSD-streamed)
+
+The **full** GLM-5.2 (744B total / 39B active) quantized to int4 with an int8 MTP head, packaged
+as the container that [**iliria**](https://github.com/PhilipJohnBasile/iliria) serves directly.
+
+iliria runs this model on a single 128 GB Apple Silicon machine by **streaming experts from the
+SSD** rather than holding all 744B parameters in memory. This repo exists so you don't have to
+run that conversion yourself — otherwise the on-ramp is downloading ~372 GB of FP8 weights and
+converting them locally for hours.
+
+## This is not the pruned build
+
+If you came from
+[GLM-5.2-Demolition-q4a4-soul-MLX](https://huggingface.co/philipjohnbasile/GLM-5.2-Demolition-q4a4-soul-MLX),
+note these are different artifacts and **not interchangeable**:
+
+| | this repo | the demolition build |
+|---|---|---|
+| model | full 744B, unpruned | expert-pruned + LoRA-healed |
+| size | ~384 GB (int4) | ~98 GB |
+| format | iliria container (`out-*.safetensors`) | MLX |
+| runs on | iliria | MLX / mlx-lm |
+
+## Requirements
+
+- Apple Silicon **M-Max / M-Ultra class with 128 GB** unified memory
+- **~384 GB free SSD** (the weights are streamed from disk, not loaded into RAM)
+- [iliria](https://github.com/PhilipJohnBasile/iliria)
+
+## Use
+
+```bash
+hf download philipjohnbasile/GLM-5.2-colibri-int4-with-int8-mtp --local-dir ./glm52-int4
+# then, from iliria:
+./illi serve --model ./glm52-int4 --ram 60
+```
+
+That exposes an OpenAI-compatible endpoint. See the iliria README for the engine's flags,
+context/RAM tuning, and measured throughput — expect deep-tier speeds, not chat speeds: this is
+a 744B model reading experts off an SSD for every token.
+
+## Contents
+
+164 `out-*.safetensors` shards (int4 experts + int8 MTP head) plus `config.json`,
+`generation_config.json`, and the tokenizer. Nothing else — no runtime state, no caches.
+
+## Provenance
+
+Quantized from [`zai-org/GLM-5.2-FP8`](https://huggingface.co/zai-org/GLM-5.2-FP8) (MIT).
+Quantization only — no pruning, no retraining, no distillation; the full expert set is intact.
+Produced by iliria's `tools/convert_fp8_to_int4.py`. Released under MIT, matching the base model.