Card: KLD vs BF16 on the fidelity suite (512 contexts)
Browse files
README.md
CHANGED
|
@@ -40,7 +40,7 @@ Community **EXL3 K2** pack of [zai-org/GLM-5.3-Flash-BF16](https://huggingface.c
|
|
| 40 |
| Spark (TP=1) | native **MTP k=2**: **15.7–16.5 tok/s** @ 8k; **14.6–15.7 tok/s** @ 64k |
|
| 41 |
| Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
|
| 42 |
| sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
|
| 43 |
-
| KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943. Method and scorer validation in the recipe's `docs/KLD.md` |
|
| 44 |
| Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
|
| 45 |
|
| 46 |
This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).
|
|
|
|
| 40 |
| Spark (TP=1) | native **MTP k=2**: **15.7–16.5 tok/s** @ 8k; **14.6–15.7 tok/s** @ 64k |
|
| 41 |
| Max ctx allocated | **131072** (KV 786,432 @ util 0.91); recommended serving ctx **65536**, see ladder |
|
| 42 |
| sixcat 0.5.1 | 120/120 think-on @ 64k — overall **84.17** on the fixed 2026-08-30 runtime (flagged: `mmlu:8` and `ifeval:1300` hit their budgets), same overall as the earlier container run |
|
| 43 |
+
| KLD vs BF16 (fidelity suite v1) | 512 contexts, 1,048,064 positions, full-vocab KL through the shared head — **token-mean 0.3346 nats** (95% CI 0.320–0.349), median 0.117, p99 3.33, **top-1 agreement 0.788**; the suite's FP8 anchor is 0.028 / 0.943; the [K2/K3 mix](https://huggingface.co/vcruz305/GLM-5.3-Flash-EXL3-K2K3-mix) scores 0.3121 / 0.795 on the same contexts. Method and scorer validation in the recipe's `docs/KLD.md` |
|
| 44 |
| Also measured | 4× L40 TP=4 text `/v1` smoke (see below) |
|
| 45 |
|
| 46 |
This is a **quantized weight pack**. Pull it, then follow the GitHub recipe for vLLM flags. Weights are MIT (Z.AI).
|