Fix streaming ONNX: audio_ctx was zero-initialized (Mimi not zero-prefix-invariant) -> streaming diverged 0.17-0.66 from offline. Now grows from empty (dynamic audio_ctx), matches offline to 0.019. Rebuild both stream_tick graphs + helper.

README.md

@@ -65,7 +65,7 @@ Four ready-to-use paths (all **fp32, same weights**). **Offline** processes a wh
 | 1 | Offline | PyTorch — `model(wav, sr)` | ~7.5 ms/frame (batch, ~11× real-time) | torch + transformers |
 | 2 | Streaming | PyTorch — `model.streaming()` | **~63 ms / 80 ms tick** (real-time, flat) | torch + transformers |
 | 3 | Offline | ONNX — `model.onnx` | ~7.3 ms/frame (~11× real-time) | onnxruntime |
-| 4 | Streaming | ONNX — `stream_tick.onnx` | **~48 ms / 80 ms tick** (real-time, flat, tightest tail) | onnxruntime |
+| 4 | Streaming | ONNX — `stream_tick.onnx` | **~52 ms / 80 ms tick** (real-time, flat, tightest tail) | onnxruntime |
 
 *Offline latency is amortized batch throughput (all frames returned after the file is processed); streaming
 latency is the real per-tick cost in a live call — both well under the 80 ms budget. Streaming is causal +
@@ -88,14 +88,14 @@ runs to keep up with one live stream (`latency × calls/sec`):
 
 | graph | infer every | latency / call | calls / audio-sec | grace / call | CPU busy / stream |
 |---|---|---|---|---|---|
-| `stream_tick.onnx` | **80 ms** | 48 ms | 12.5 | 32 ms | ~60% |
-| `stream_tick_240.onnx` | **240 ms** | 80 ms | 4.17 | 160 ms | ~33% |
+| `stream_tick.onnx` | **80 ms** | 52 ms | 12.5 | 28 ms | ~65% |
+| `stream_tick_240.onnx` | **240 ms** | 85 ms | 4.17 | 155 ms | ~36% |
 
-- **80 ms — snappiest, but heaviest.** Each call eats 48 ms → only **32 ms grace**, pool busy **~60%** per
+- **80 ms — snappiest, but heaviest.** Each call eats 52 ms → only **28 ms grace**, pool busy **~65%** per
   stream. Fastest reactions, least room for anything else.
-- **240 ms — much cheaper, lots of slack.** Each call takes 80 ms (vs 3 × 48 = 144 ms for three 80 ms ticks)
-  but leaves **160 ms grace** and sits at **~33% busy**.
-- **The real win is at scale.** CPU per second of audio drops **~1.8×** (600 → 332 ms/s, ≈45% less), so **one
+- **240 ms — much cheaper, lots of slack.** Each call takes 85 ms (vs 3 × 52 = 156 ms for three 80 ms ticks)
+  but leaves **155 ms grace** and sits at **~36% busy**.
+- **The real win is at scale.** CPU per second of audio drops **~1.8×** (650 → 354 ms/s, ≈45% less), so **one
   machine holds roughly ~1.8× as many concurrent calls** — the per-stream % looks modest, but that factor
   multiplies across every simultaneous call in your fleet. *(`CPU busy %` is a single-stream duty cycle
   measured with 4 threads — read it as relative cost between graphs, not absolute single-core load.)*
@@ -187,7 +187,7 @@ graph is fully self-contained — no `torch` or `transformers` needed at inferen
 
 Torch-free real-time streaming via `stream_tick.onnx` + the `onnx_streaming.py` helper (`DualTurnONNXStreamer`).
 Each 80 ms tick only computes the **new frame** — the graph carries all state (conv buffer, transformer **KV
-window**, downsample history, LSTM state) in/out, so latency is low (**~48 ms/tick**) and **flat over long
+window**, downsample history, LSTM state) in/out, so latency is low (**~52 ms/tick**) and **flat over long
 calls**. Decision-equivalent to the PyTorch model.
 
 ```python

onnx_streaming.py

@@ -48,8 +48,8 @@ class _BaseStreamer:
 
     def reset(self):
         z = np.zeros
-        self.state = {"audio_ctx": z((2, self.RF), np.float32),
-                      "pk": z((self.NL, 2, self.H, 0, self.HD), np.float32),   # KV starts empty, grows, capped in-graph
+        self.state = {"audio_ctx": z((2, 0), np.float32),                       # EMPTY: grow from true start (no
+                      "pk": z((self.NL, 2, self.H, 0, self.HD), np.float32),   #  zero-prefix; graph caps at RF). KV likewise grows/caps in-graph
                       "pv": z((self.NL, 2, self.H, 0, self.HD), np.float32),
                       "tf_hist": z((2, self.TFK, 512), np.float32),
                       "h": z((2, 1, 256), np.float32), "c": z((2, 1, 256), np.float32),

stream_tick.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:737418bae8eec4321f9f72120eb2250739f230e1d087e01b469d0beebe80ef7d
-size 161721665
+oid sha256:6700ea2f919b4c66355b3dafb4f91f2db9dae9e18a302b812730c04340bf819f
+size 161722822

stream_tick_240.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b406cd4ced3fb6f9614713fc742e23bd3da8b0a498ced5a9aa39a7d80ac9bd14
-size 162399971
+oid sha256:e841d80d2a004ab4ea9d2e85f168505853064c4a9ba7341d180cf352062ed7c7
+size 162401094