vcruz305 commited on
Commit
ca0bcda
·
verified ·
1 Parent(s): ed7681b

Card: measured long-context ceiling (verified to 163k prompt tokens; 262k boots)

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -41,6 +41,7 @@ Community **EXL3 K2** pack of [zai-org/GLM-5.3-Flash-BF16](https://huggingface.c
41
  | Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
42
  | sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
43
  | KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943; the [K2/K3 mix](https://huggingface.co/vcruz305/GLM-5.3-Flash-EXL3-K2K3-mix) scores 0.3121 / 0.795 on the same contexts. Method and scorer validation in the recipe's `docs/KLD.md` |
 
44
  | Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
45
 
46
  This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).
 
41
  | Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
42
  | sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
43
  | KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943; the [K2/K3 mix](https://huggingface.co/vcruz305/GLM-5.3-Flash-EXL3-K2K3-mix) scores 0.3121 / 0.795 on the same contexts. Method and scorer validation in the recipe's `docs/KLD.md` |
44
+ | long context | boots at **262,144** (KV pool 1,093,332 tokens, 4.17x full-256k concurrency); real-text needle ladder verified to **163,479 prompt tokens** (needle recalled verbatim, prefill ~590 tok/s, decode 17–20). Known runtime bug: prompts between ~163k and ~180k wedge the engine — keep prompts ≤163k until the fix lands; ≤131,072 contexts unaffected. Details in the recipe README |
45
  | Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
46
 
47
  This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).