GLM-5.3-NVFP4-AQLM-hybrid

GLM-5.3 compressed to fit 384 GB VRAM by a two-tier per-expert scheme, same recipe and checkpoint schema as jarrelscy/GLM-5.2-NVFP4-AQLM-hybrid:

• Hot experts (7100 of 19,200 = 37%): donor NVFP4 (4.5 bpw effective), chosen per layer by routing mass over a 15M-token calibration set (floor 8 / cap 176 per layer; hot experts carry 46–62% of routing mass).
• Cold experts: 2 bpw AQLM (1×16 codebook, group 8), k-means init + activation-aware CD re-encode against the donor teacher (diag-Hessian, per-layer weighted rel-err after converge: w13 ≈ 0.085, w2 0.0003–0.08).
• MTP layer 78: all 256 experts requantized BF16→NVFP4 (rel-frob ≈ 0.095); input_scale synthesized from the 5.2 L77→L78 ratio (validate MTP acceptance rate when serving).
• Non-expert weights: donor NVFP4/BF16 passthrough. Total 287.5 GiB.

Vision

The GLM-5.2V vision tower + projector (from baseten/GLM-5.2-Vision-NVFP4, byte-identical) are grafted on: vision_tower.safetensors + mm_projector.safetensors, glm5v config, Kimi-K2.5-style processor. The encoder was trained against the frozen 5.2 text tower; 5.2→5.3 drift is small (mean embedding row-cos 0.992). An A100 probe (projected image tokens spliced at <|image|>, teacher-forced caption CE through this hybrid) shows correct caption preference both ways with wide margins. Fine-grained VQA/OCR quality vs 5.2V has not yet been benchmarked.

Quality (teacher-forced ppl, non-overlapping 2048-token windows, A100 bf16-dequant)

set	BF16-dequant donor	this hybrid	Δ
heldout	2.9002	2.9763	+2.6%
heldout-xl	2.6047	2.7199	+4.4%
wikitext-2	3.0150	3.6094	+19.7%
github (post-2026-06)	2.7469	2.7878	+1.5%

Status: post-converge, pre-PV. The wikitext gap reflects hot/cold assignment bias toward the code/technical-heavy calibration set; a REAP saliency retier + PV tune pass is in progress and this repo will be updated in place.

Serving

Same stack as the 5.2 hybrid: an sglang/vLLM build with the nvfp4_aqlm_hybrid quant method (per-layer hyb_kind routes each expert to the NVFP4 or AQLM dequant path) and the glm5v wrapper (MoonViT tower + PatchMerger projector + GlmMoeDsa text backbone).

Dequant checks: hot experts bit-exact vs donor NVFP4; cold codes bit-exact vs the converge output (SC-2 max|d| = 0).