sync: latest measured numbers
Browse files- RESULTS.md +10 -0
RESULTS.md
CHANGED
|
@@ -46,3 +46,13 @@ DSA tilelang, NVFP4) DFlash2-on is NOT bit-identical to DFLASH-off at temp 0.**
|
|
| 46 |
drafter card's "greedy output matches the target exactly" does not reproduce here; whether
|
| 47 |
the cause is verify-path numerics on this chip or quant interaction is unresolved. Users
|
| 48 |
needing bit-exact greedy reproducibility should serve DFLASH-off.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
drafter card's "greedy output matches the target exactly" does not reproduce here; whether
|
| 47 |
the cause is verify-path numerics on this chip or quant interaction is unresolved. Users
|
| 48 |
needing bit-exact greedy reproducibility should serve DFLASH-off.
|
| 49 |
+
|
| 50 |
+
## Thinking mode vs effort (2026-08-28)
|
| 51 |
+
Decode tok/s is IDENTICAL with thinking on or off (~18-22 on this probe; run variance
|
| 52 |
+
exceeds any mode difference). But with a tight max_tokens budget, thinking-ON can spend
|
| 53 |
+
the ENTIRE budget on reasoning and return zero answer: at 600 max_tokens our probe got
|
| 54 |
+
2,600 chars of reasoning_content and empty content (finish=length), while thinking-OFF
|
| 55 |
+
returned 2,462 chars of pure answer in the same wall time. For agent/tool workloads,
|
| 56 |
+
disable thinking per-request ("chat_template_kwargs": {"enable_thinking": false}) or
|
| 57 |
+
budget max_tokens for reasoning + answer. Conditions: merge-sorted-lists code prompt,
|
| 58 |
+
600 max_tokens, temp 0, FP8T8V config.
|