randomllama commited on
Commit
498eadf
·
verified ·
1 Parent(s): d72ca7c

sync: latest measured numbers

Browse files
Files changed (1) hide show
  1. LADDER.md +15 -0
LADDER.md CHANGED
@@ -118,3 +118,18 @@ states shaving accept length; deep-batch accept len ~3.1 vs ~4.2 single). Correc
118
  PASS, no token-0 collapse. **Production is now FP8T8b** — fp8-KV prefill wins AND the concurrency
119
  curve, one config. (c-sweep prompts: 12 distinct short code/infra prompts, 400 max_tokens, temp 0,
120
  stream:false, warmed; clocks stock.)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118
  PASS, no token-0 collapse. **Production is now FP8T8b** — fp8-KV prefill wins AND the concurrency
119
  curve, one config. (c-sweep prompts: 12 distinct short code/infra prompts, 400 max_tokens, temp 0,
120
  stream:false, warmed; clocks stock.)
121
+
122
+ ## Vision unlock (2026-08-28 pm) — GLM-5.3-Flash is MULTIMODAL, and it works on this stack
123
+ Correction of our own record: GLM-5.3-Flash has a full 24-layer vision tower with image AND
124
+ video tokens (upstream config: Glm5NextForConditionalGeneration). The LibertAIDAI NVFP4 quant
125
+ ships all 347 model.visual.* tensors, and the SGLang #36507 branch implements the vision path.
126
+ Our deployments simply never passed --enable-multimodal.
127
+
128
+ Config FP8T8V = FP8T8b + `--enable-multimodal`. Results:
129
+ - vision gate: PASS (64x64 solid-red data-URL probe answered "Red", temp 0)
130
+ - single-stream: 29.3 code / 23.8 prose — no measurable vision-tower tax (matches best fp8)
131
+ - c-sweep: c1 34.5 / c2 51.1 / c4 44.5 / c8 78.7 (8/8) / c12 79.5 — concurrency intact
132
+ (c12 delta vs FP8T8b's 85.4 is single-run noise territory; c4 dip reproduces in both)
133
+ **Production is now FP8T8V: fp8-KV + 8 concurrent streams + image input, one config.**
134
+ Not yet measured: vision quality beyond the smoke probe, video input, vision+DFlash accept
135
+ interaction, vision under concurrency. Treat image support as verified-working, not benchmarked.