---
license: mit
base_model: zai-org/GLM-5.3-Flash
tags: [reap, moe, pruning, glm5_next, jetson, thor]
---

# GLM-5.3-Flash REAP-50 (FP8)

> ## Corrected 2026-08-28 — healing scalar re-fit
>
> **This checkpoint has been revised.** The original upload applied a first-moment healing gain
> (median **0.6933**) to every retained expert's output. That estimator ignores that
> `norm_topk_prob` renormalises the surviving top-8, so the router already returns essentially all
> the pruned gate mass by itself — **measured: gate mass 2.5000 before pruning and 2.5000 after**.
> The derivation therefore prescribed a ~39% shrink where the true output inflation is ~10%.
>
> Re-measured by replaying post-prune routing from a cached router-score trace over 2.3M tokens,
> the correct gain is median **0.9111** (per-layer 0.846–0.950).
> Every layer's block scales have been multiplied by `measured / shipped`
> (median **1.3121**, range 1.0719–1.5260), correcting a systematic
> under-scaling of the entire MoE pathway relative to attention, the shared experts and the
> residual stream.
>
> The fix is exact and lossless: it scales the F32 `weight_scale_inv` tensors, never the FP8
> values, so no weight was requantised. 6,048 block scales were corrected — exactly
> 42 layers x 144 retained experts.
>
> **If you downloaded the earlier revision, re-pull.** The previous state remains in this repo's
> git history, and is recoverable by dividing the block scales by the factors above.
>
> Not yet propagated to the NVFP4 sibling repo, which still carries the original gain.


50% of routed experts removed with **REAP**
(Router-weighted Expert Activation Pruning, arXiv:2510.13999), calibrated on a
permissively-licensed multi-domain corpus that includes real image-text pairs.

| | |
|---|---|
| Base | `zai-org/GLM-5.3-Flash` (MIT, FP8 E4M3, 128x128 block scales) |
| Experts | 288 -> 144 per layer, top-8 routing unchanged |
| Size | 157.0 GiB (FP8) |
| Healed | yes — **re-fit 2026-08-28**, measured gain 0.9111 |
| MTP block | excluded (see below) |

## Why FP8 and not BF16

The upstream release is **FP8**, not BF16. Routed experts are stored per-expert with their own
`weight_scale_inv` block scales, so pruning is deleting whole tensors - **lossless on every
retained weight**. The 642 GB BF16 repo elsewhere on the Hub is a dequantised upcast carrying
no additional information.

## Calibration

Mixture weighted for a coding/agentic model that stays empirically grounded: agentic 24%,
code 21%, math 15%, multimodal 15%, science+bio 10%, finance 8%, ballast 7%.
Permissive licences only, so this checkpoint keeps the base model's MIT lineage.

Vision is first-class: the vision tower contains no MoE and is untouched, but image tokens
route through the same expert pool as text, so text-only calibration would have deleted
vision-serving experts with certainty. Real image-text pairs were asserted present.

## Evaluation status: NONE

**This checkpoint has not been evaluated.** No benchmark has been run against it - not coding,
not agentic, not vision, not knowledge. What has been verified is *structural*: expert counts
match the config, routers are sliced to the retained set, every tensor loads, the vision tower
is untouched, and the MTP block is cleanly absent.

The pruning itself measured **1.29x better than random** at retaining expert output
contribution (saliency mass 0.643 against 0.50 for random pruning at the same ratio). That says
the criterion selected well. It does **not** say the model is good.

Treat this as a research artifact pending evaluation, not a drop-in replacement.

## Known limitations

- **The MTP (multi-token-prediction) block at layer 45 is excluded.** `transformers`'
  `Glm5NextForConditionalGeneration` does not instantiate it, so the pruning path cannot see
  it. Its original tensors are archived unmodified rather than inconsistently pruned.
- REAP has no published data above 50% compression; this checkpoint sits at the validated
  ceiling, not beyond it.
- Expect **factual-recall** regression before reasoning or coding regression. That is the
  measured failure mode of expert pruning on this architecture family: the closest published
  analogue (`cerebras/Kimi-Linear-REAP-35B-A3B`, same KDA + full-attention stack) loses 3.4
  points on FRAMES at only 30% pruning while code and maths hold flat.
- Healing is a **first-moment output-scale correction** derived from the calibration saliency
  (median gain 0.696, applied exactly to the F32 block scales). It is *not* distillation and
  does not attempt to recover lost knowledge.
- Routing is disrupted more than expert count suggests: the retained experts carry ~0.90x the
  routing mass an average expert would, because REAP preserves rare-but-strong experts over
  common-but-weak ones.

## Serving on Jetson Thor

Use the **cutlass** fused-MoE backend (the Marlin FP4 MoE kernel faults at >=256 experts) and
`TRITON_MLA` for the 11 MLA+DSA layers (FLASHINFER is invalid for MLA).