---
language:
- en
library_name: mlx
license: mit
pipeline_tag: image-text-to-text
base_model: zai-org/GLM-5.3-Flash
tags:
- mlx
- jang
- quantized
- apple-silicon
- vision
- video
- reasoning
- agent
- tool-use
- glm5_next
- moe
- imatrix
- awq
---

<p align="center">
  <img src="./jangq-logo.png" alt="JANGQ" width="220">
  &nbsp;&nbsp;&nbsp;
  <img src="./vmlx-logo.png" alt="vMLX" width="90">
</p>

> ⚠️ **Runtime not ready yet.** These bundles use a new architecture
> (`glm5_next`: KDA linear attention + MLA/DSA hybrid + mHC) — vMLX
> Swift/Python runtime support is in active development and NOT released.
> Access is gated until it lands. Nothing loads these correctly today
> except the internal evaluation runtime they were built and measured with.

# JANGQ-AI/GLM-5.3-Flash-JANG-MTP

**GLM-5.3-Flash for 128 GB Macs — with the native multi-token-prediction layer preserved for self-speculative decode (~1.5–2× expected once runtime support lands).**

A JANG bundle of [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) — 300B-class MoE (288 experts, top-8 + shared) with KDA linear attention, sparse attention, and vision+video towers — quantized for Apple Silicon / MLX with a fully measured, per-unit dynamic bit allocation. Attention, routing, and all gating parameters are kept at 8-bit or full precision; every low-bit byte lives in the routed experts, placed by measurement.

Sibling bundle: [GLM-5.3-Flash-JANG](https://huggingface.co/JANGQ-AI/GLM-5.3-Flash-JANG)

## Quality — measured, not estimated

15,850 teacher-forced positions on held-out prompts, versus the official FP8 release's logits (top-128, renormalized):

| Bundle | Size | median KL | mean KL | p90 / p95 / p99 | top-1 | top-5 | top-10 |
|---|---|---|---|---|---|---|---|
| **JANG-MTP** | **95.47 GiB** | **0.0977** | **0.552** | **1.57 / 2.63 / 5.74** | **78.1%** | **94.4%** | **96.7%** |
| JANG (AR) | 95.48 GiB | 0.0885 | 0.529 | 1.50 / 2.56 / 5.64 | 78.7% | 94.6% | 96.9% |

For calibration: our dots3-note release (280B at 94.6 GiB) ships at 79.2% top-1 — this model carries ~305B of routed experts with no fp16 embedding table to absorb signal, so these numbers are the honest physics of ~2.2 effective bits at this size.

## How it compares (same protocol, same reference, same positions)

| Quant (95 GiB class) | Size | median KL ↓ | mean KL ↓ | top-1 ↑ | top-5 ↑ | top-10 ↑ |
|---|---|---|---|---|---|---|
| **GLM-5.3-Flash-JANG (AR)** | 95.35 GiB | **0.0885** | **0.52** | **78.7%** | **94.6%** | **96.8%** |
| **GLM-5.3-Flash-JANG-MTP** | 95.47 GiB | 0.0977 | 0.55 | 78.1% | 94.4% | 96.7% |
| orcarouter GLM-5.3-Flash-MLX `2bit-lite` | 95.4 GiB | 0.2122 | 0.83 | 71.4% | 90.8% | 94.3% |

All rows: 15,850 teacher-forced positions vs the official FP8 release
(top-128 renormalized KL). The orcarouter bundle was evaluated by loading its
quantized weights natively — its exact shipped fidelity, no requantization.
antirez's `GLM-5.3-Flash-Q2.gguf` (89.9 GiB) could not be measured: no public
llama.cpp build (mainline or the open support PR) currently loads those files.

## Calibration data

600,064 calibration tokens — 50% web text, 25% code, 15% multi-turn chat (incl. tool-call transcripts), 10% math/reasoning — with evaluation prompts drawn from a disjoint held-out tail. Activation-aware scaling and a per-channel importance refit are applied throughout; per-expert statistics cover 285–288 of 288 experts per layer.

## What's in the bundle

- **Vision + video**: full tower (8-bit) + the consolidated image/video processor config.
- **MTP**: the native multi-token-prediction layer is preserved (shares the sparse-attention indexer). Draft depth requires a measured sweep on the target runtime.
- **Thinking + agentic**: thinking ON by default (the template force-opens `<think>`), reasoning efforts `low / high / max` (default **max**), `clear_thinking=false` preserves history thinking. Tool calls use GLM's XML dialect (`<tool_call>name<arg_key>…<arg_value>…`) — declared as `tool_parser: glm_xml_args` in the config; Hermes-style JSON parsers will not work.
- **Self-describing quantization**: per-module `quantization` block (bits + group size for all 626 quantized modules) in `config.json`.
- **128 GB Mac fit**: ~95.5 GiB weights with a uniquely small cache footprint (fixed-size linear-attention state + compressed-latent KV ≈ 6 KB/token) — long contexts do not balloon memory.

## Serving contract

- Sampling: `temperature=1.0, top_p=0.95` (vendor defaults)
- EOS: `[154820, 154827, 154829]` · context: 1M native
- Reasoning: efforts low/high/max via `reasoning_effort` chat-template kwarg, default max; `clear_thinking` strips history thinking when true
- Tools: `glm_xml_args` dialect; tool results render as `<|observation|><tool_response>…`

Quantized and validated by **Jinho Jang** — eric@jangq.ai