randomllama commited on
Commit
e91cfbc
·
verified ·
1 Parent(s): b8a968e

sync: latest measured numbers

Browse files
Files changed (1) hide show
  1. LADDER.md +18 -0
LADDER.md CHANGED
@@ -71,3 +71,21 @@ Campaign complete. Closeout data in RESULTS.md; morning package in MORNING.md.
71
  ~55 aggregate is the machine's bandwidth plateau either way. **Production config
72
  changed to DFlash + max_running 8** (best latency at every c<8, ~94% of the fleet
73
  ceiling at c12).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  ~55 aggregate is the machine's bandwidth plateau either way. **Production config
72
  changed to DFlash + max_running 8** (best latency at every c<8, ~94% of the fleet
73
  ceiling at c12).
74
+
75
+ ## fp8-KV tilelang CUDA port (2026-08-28) — WORKS ON GB10
76
+ The gate was plumbing, not kernels: a complete raw-fp8 sparse kernel (`sparse_mla_fwd_decode_partial_fp8`)
77
+ ships in-tree, HIP-gated in 3 plumbing sites (dispatch, pool layout, fused-quant write). Port = relax the
78
+ gate (CUDA + both-backends-tilelang + SM89+), route the raw 512 B/token layout on CUDA, key the raw write
79
+ on layout not platform. Kernel verified by execution on sm_121 first (one-hot exact 0.000000; negative
80
+ control fails at 91%; fp8 GEMMs lower and run).
81
+ | check | result |
82
+ |---|---|
83
+ | boot-gate negative control (mixed backends + fp8) | **died as required** (ValueError) |
84
+ | point-of-effect log line | present (stock code cannot print it) |
85
+ | 19×21 gate / token-0 collapse | PASS / none |
86
+ | decode (n=5 medians) | 29.2 code / 22.9 prose — **parity** with same-image bf16 (29.8) |
87
+ | temp-0 vs bf16, 5 prompts | **4/5 exact** |
88
+ | 32K-depth codeword probe | PASS |
89
+ | TTFT@16k warm | **6.6 s vs 7.9 s bf16 — 17% faster prefill** |
90
+ | open item | raw-layout memory accounting resolved max_running_requests to 1 — budget retune WIP |
91
+ Kernel analysis + port: this rig (Fable agent), validated per the house protocol.