Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,138 @@
+---
+license: apache-2.0
+license_name: mixed-per-model
+license_link: LICENSE
+library_name: onnx
+pipeline_tag: audio-classification
+language:
+  - en
+tags:
+  - end-of-turn-detection
+  - turn-taking
+  - voice-agents
+  - speech
+  - onnx
+  - from-scratch
+datasets:
+  - pipecat-ai/smart-turn-data-v3.2-train
+  - Scicom-intl/semantic-vad-eot
+  - li2017dailydialog/daily_dialog
+metrics:
+  - roc_auc
+  - average_precision
+model-index:
+  - name: TurnWave
+    results:
+      - task:
+          type: audio-classification
+          name: End-of-turn detection
+        dataset:
+          type: livekit/eot-bench-data
+          name: eot-bench (English)
+          split: validation
+        metrics:
+          - type: roc_auc
+            value: 0.77
+            name: AUC
+          - type: average_precision
+            value: 0.602
+            name: Average precision
+          - type: false_cutoff_rate
+            value: 42.1
+            name: False cutoffs @300ms latency budget (%)
+          - type: false_cutoff_rate
+            value: 17.2
+            name: False cutoffs @600ms latency budget (%)
+---
+
+# TurnWave — end-of-turn detection for voice agents
+
+Decides whether a caller has **finished speaking** or is only pausing, so a voice
+agent neither interrupts them nor leaves an awkward silence. It replaces the fixed
+300–700 ms silence timeout most pipelines still use.
+
+**Trained from scratch — no pretrained weights anywhere.** A causal transformer
+(RoPE, RMSNorm, SwiGLU) over the transcript tail, and a CNN over log-mel
+spectrograms for prosody. Even the log-mel front end is hand-built on `torch.stft`,
+so there is no torchaudio or librosa dependency.
+
+## Benchmark
+
+Scored by [LiveKit's eot-bench](https://github.com/livekit/eot-bench) harness on
+real human-to-agent conversation, using their code and published baselines. Lower
+is better; **bold marks the best per column.**
+
+| model | false cutoffs @300 ms ↓ | @600 ms ↓ | latency @5% cutoff ↓ |
+|---|---|---|---|
+| VAD baseline | 55.6% | 21.7% | 1600 ms |
+| **TurnWave audio branch (this model)** | 42.1% | 17.2% | 1195 ms |
+| SmartTurn v3.2 | 35.2% | 14.8% | 1051 ms |
+| LiveKit Turn Detector v1 | **9.9%** | **4.5%** | **543 ms** |
+
+TurnWave beats the VAD baseline on every metric the harness reports.
+
+## Models in this repo
+
+| file | licence | training data |
+|---|---|---|
+| `audio_eot_v2.onnx` | apache-2.0 | trained on smart-turn conversational clips |
+| `audio_eot.onnx` | cc-by-4.0 | Phase 4; trained on semantic-vad-eot (CC BY 4.0) |
+| `text_eot.int8.onnx` | cc-by-nc-sa-4.0 | trained on DailyDialog (CC BY-NC-SA 4.0) — non-commercial |
+| `fusion_eot.onnx` | cc-by-nc-sa-4.0 | contains the text branch, so it inherits the same terms |
+
+Each model's licence follows its training data, so they differ. `audio_eot_v2` is
+the one the benchmark above measures and the one to use.
+
+## Usage
+
+```python
+from huggingface_hub import hf_hub_download
+from turnwave.infer import TurnDetector   # pip install git+https://github.com/Nikhils-G/turnwave
+
+detector = TurnDetector(hf_hub_download("Nikhil-09/turnwave", "audio_eot_v2.onnx"))
+if detector.predict(audio=wav_16k) > 0.5:
+    respond()
+```
+
+16 kHz mono. The model reads the last 2 seconds ending at the decision point, which
+sits 0.2 s into the pause — where a live agent decides, and where eot-bench scores.
+
+| model | variant | CPU latency | size |
+|---|---|---|---|
+| audio_eot_v2 | fp32 | 4.81 ms | 14.0 MB |
+| audio_eot | fp32 | 4.52 ms | 14.0 MB |
+| text_eot | int8 | 5.02 ms | 7.2 MB |
+| fusion_eot | fp32 | 9.35 ms | 42.4 MB |
+
+INT8 is not applied blindly: dynamic quantization rewrites MatMul, so it speeds up
+the transformer and *slows down* the conv-heavy branches. Each model ships whichever
+variant measured faster.
+
+## What this project found
+
+The first version of this model scored **AP 0.945** on its own held-out test set and
+**AUC 0.563** on eot-bench — barely above random. The policy sweep chose thresholds
+of 0.0 and 1.0, meaning *ignore the model entirely*.
+
+The cause was the training corpus, not the architecture. It derived from a dataset
+whose own card declares `task_categories: [text-to-speech]` — read speech, whose
+pauses are reading hesitations rather than conversational turn-yields. The model had
+learned *"has this sentence finished being read aloud."*
+
+Retraining on conversational data, changing nothing else, lifted AUC to **0.770**.
+The in-domain score could never have revealed this; only a benchmark on data we did
+not build could.
+
+## Limitations
+
+- **English only.** Other languages are in the training data but untested here.
+- **Behind the production models**, and not a fair comparison: SmartTurn starts from
+  a pretrained Whisper encoder, LiveKit's is a fine-tuned 0.5B LLM distilled from a
+  7B teacher. This is 3.49M parameters from random initialisation.
+- **The fusion model is stale.** It was trained on the read-speech corpus, which the
+  benchmark showed to be the wrong task. The conversational corpus has no
+  transcripts, so retraining fusion needs ASR first.
+- **Non-commercial models included.** The text and fusion models derive from
+  DailyDialog (CC BY-NC-SA 4.0). Only the audio branches are permissively licensed.
+
+Code, training scripts, and the full write-up: **https://github.com/Nikhils-G/turnwave**