ManniX-ITA commited on
Commit
0bb249d
·
verified ·
1 Parent(s): 3f0e8f3

docs: PolyKV stress benchmark section (#742)

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -119,6 +119,33 @@ unzip -p opencoti-llamafile-*.llamafile ggml-cuda.so | sha256sum
119
  Apache-2.0 (llamafile) + MIT (llama.cpp and bundled projects).
120
  opencoti patches and packaging are MIT.
121
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122
  ---
123
 
124
  <img src="vodafone-logo.png" alt="Vodafone" width="96"/>
 
119
  Apache-2.0 (llamafile) + MIT (llama.cpp and bundled projects).
120
  opencoti patches and packaging are MIT.
121
 
122
+
123
+ ## PolyKV multi-session stress benchmark (package_courier)
124
+
125
+ An agentic multi-session benchmark: LangGraph courier agents (tool-calling,
126
+ 10-step delivery chains, 100 packages) run against ONE engine at
127
+ `--parallel 32`, ctx 163840, q8_0 KV, with a tps-floor scheduler that admits
128
+ new sessions only while every live session keeps ≥ floor tok/s (PolyKV
129
+ `/polykv/tps` SSE telemetry drives admission). Harness source:
130
+ `opencoti/langgraph/bench/package_courier/` in the repo; reports under
131
+ [`bench/reports/`](bench/reports/) (per-run HTML + raw stats).
132
+
133
+ | model (MTP) | floor | score | wall | delivery mean | fan-out |
134
+ |---|---|---|---|---|---|
135
+ | Qwen3.6-27B-Omnimerge-v4 Q6_K (NextN self-spec) | 15 | 99/100 | 2171 s | 146 s | ~7 |
136
+ | ″ | 5 | 99/100 | 2509 s | 467 s | →22 |
137
+ | ″ | 1 | 98/100 | 2803 s | 818 s | 32 (max) |
138
+ | Gemma-4 26B-A4B-it Q4_K_M (assistant-MTP) | 15 | 100/100 | 864 s | 51 s | med 6 |
139
+ | ″ | 5 | 99/100 | 1090 s | 145 s | med 14 |
140
+ | ″ | 1 | 95/100 | 1340 s | 346 s | 32 (max) |
141
+
142
+ Aggregate engine throughput stays pinned (Qwen ~75-80 tok/s regardless of
143
+ concurrency), so equilibrium workers ≈ aggregate/floor. The A4B + assistant-MTP
144
+ pair is ~2.5× faster end-to-end at every rung. All losses across all runs are
145
+ agent-side (turn-budget wander on hard packages) — zero tool errors, zero LLM
146
+ errors, engines clean throughout. The `ab-pooled` vs `ab-naive` pair isolates
147
+ the SharedKVPool win at identical workload.
148
+
149
  ---
150
 
151
  <img src="vodafone-logo.png" alt="Vodafone" width="96"/>