Card: measured long-context ceiling (verified to 163k prompt tokens; 262k boots)
Browse files
README.md
CHANGED
|
@@ -41,6 +41,7 @@ Community **EXL3 K2** pack of [zai-org/GLM-5.3-Flash-BF16](https://huggingface.c
|
|
| 41 |
| Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
|
| 42 |
| sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
|
| 43 |
| KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943; the [K2/K3 mix](https://huggingface.co/vcruz305/GLM-5.3-Flash-EXL3-K2K3-mix) scores 0.3121 / 0.795 on the same contexts. Method and scorer validation in the recipe's `docs/KLD.md` |
|
|
|
|
| 44 |
| Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
|
| 45 |
|
| 46 |
This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).
|
|
|
|
| 41 |
| Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
|
| 42 |
| sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
|
| 43 |
| KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943; the [K2/K3 mix](https://huggingface.co/vcruz305/GLM-5.3-Flash-EXL3-K2K3-mix) scores 0.3121 / 0.795 on the same contexts. Method and scorer validation in the recipe's `docs/KLD.md` |
|
| 44 |
+
| long context | boots at **262,144** (KV pool 1,093,332 tokens, 4.17x full-256k concurrency); real-text needle ladder verified to **163,479 prompt tokens** (needle recalled verbatim, prefill ~590 tok/s, decode 17–20). Known runtime bug: prompts between ~163k and ~180k wedge the engine — keep prompts ≤163k until the fix lands; ≤131,072 contexts unaffected. Details in the recipe README |
|
| 45 |
| Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
|
| 46 |
|
| 47 |
This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).
|