randomllama commited on
Commit
2c625d4
·
verified ·
1 Parent(s): da788b0

sync: latest measured numbers

Browse files
Files changed (1) hide show
  1. RESULTS.md +18 -0
RESULTS.md CHANGED
@@ -114,3 +114,21 @@ Scope caveat: a 44-request short-form matrix rules out gross corruption, not rar
114
  corruption; the #36885 sentinel bug is claimed load-dependent. Methodology note: our first run
115
  "failed" 7/44 — every failure was reasoning-narration truncated by an 80-token cap, zero wrong
116
  values; budget raised to 400 and the checker's false accusations vanished. Check your checker.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
  corruption; the #36885 sentinel bug is claimed load-dependent. Methodology note: our first run
115
  "failed" 7/44 — every failure was reasoning-narration truncated by an 80-token cap, zero wrong
116
  values; budget raised to 400 and the checker's false accusations vanished. Check your checker.
117
+
118
+ ## SHOOTOUT round 2 (2026-08-28 late) — EXL3 lane rev 1df71c1 (MiaAI-Lab's concurrency+context update)
119
+ Re-ran after MiaAI-Lab shipped 7 commits (DFlash2/MLA KV page-sharing, 1M default context,
120
+ prefix-cache fix). Same unified protocol, our rig, their new image. What changed:
121
+ | metric | EXL3 v1 (bd7f55e) | EXL3 v2 (1df71c1) | SGLang fp8 (ours) |
122
+ |---|---:|---:|---:|
123
+ | code c1 | 51.9 | 45.3 | 29.3 |
124
+ | structured | 51.3 | 51.1 | 43.3 |
125
+ | prose | 23.7 | 25.0 | 29.2 |
126
+ | c8 aggregate | 63.8 | 57.7 | 78.1 |
127
+ | c12 aggregate | 53.3 | 59.9 | 83.5 |
128
+ | 54k-token prefill | (not tested) | **PASS** (worker survived, 119GB peak) | **worker dies ~62k** |
129
+ Read: her concurrency fix is real but partial — c12 improved +12% (53.3→59.9) yet still trails
130
+ our 83.5 and still degrades past c4. Her decisive new edge is LONG CONTEXT reliability: her lane
131
+ recalled a needle at 34k AND 54k tokens with the worker surviving, where our stack silently kills
132
+ rank1 at ~62k (the blocker from our night-3 probe). Her MLA/DFlash2 KV page-sharing is doing what
133
+ her commits claim. Honest scorecard now: EXL3 wins solo code + long context; SGLang fp8 wins prose
134
+ + fleet concurrency. Both re-measured same rig, same night. Credit MiaAI-Lab for a fast, real fix.