sync: latest measured numbers
Browse files
LADDER.md
CHANGED
|
@@ -118,3 +118,18 @@ states shaving accept length; deep-batch accept len ~3.1 vs ~4.2 single). Correc
|
|
| 118 |
PASS, no token-0 collapse. **Production is now FP8T8b** — fp8-KV prefill wins AND the concurrency
|
| 119 |
curve, one config. (c-sweep prompts: 12 distinct short code/infra prompts, 400 max_tokens, temp 0,
|
| 120 |
stream:false, warmed; clocks stock.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
PASS, no token-0 collapse. **Production is now FP8T8b** — fp8-KV prefill wins AND the concurrency
|
| 119 |
curve, one config. (c-sweep prompts: 12 distinct short code/infra prompts, 400 max_tokens, temp 0,
|
| 120 |
stream:false, warmed; clocks stock.)
|
| 121 |
+
|
| 122 |
+
## Vision unlock (2026-08-28 pm) — GLM-5.3-Flash is MULTIMODAL, and it works on this stack
|
| 123 |
+
Correction of our own record: GLM-5.3-Flash has a full 24-layer vision tower with image AND
|
| 124 |
+
video tokens (upstream config: Glm5NextForConditionalGeneration). The LibertAIDAI NVFP4 quant
|
| 125 |
+
ships all 347 model.visual.* tensors, and the SGLang #36507 branch implements the vision path.
|
| 126 |
+
Our deployments simply never passed --enable-multimodal.
|
| 127 |
+
|
| 128 |
+
Config FP8T8V = FP8T8b + `--enable-multimodal`. Results:
|
| 129 |
+
- vision gate: PASS (64x64 solid-red data-URL probe answered "Red", temp 0)
|
| 130 |
+
- single-stream: 29.3 code / 23.8 prose — no measurable vision-tower tax (matches best fp8)
|
| 131 |
+
- c-sweep: c1 34.5 / c2 51.1 / c4 44.5 / c8 78.7 (8/8) / c12 79.5 — concurrency intact
|
| 132 |
+
(c12 delta vs FP8T8b's 85.4 is single-run noise territory; c4 dip reproduces in both)
|
| 133 |
+
**Production is now FP8T8V: fp8-KV + 8 concurrent streams + image input, one config.**
|
| 134 |
+
Not yet measured: vision quality beyond the smoke probe, video input, vision+DFlash accept
|
| 135 |
+
interaction, vision under concurrency. Treat image support as verified-working, not benchmarked.
|