README.md

dualturn-endpointing

Speech endpoint detection for two-channel (user + agent) audio.

Answers one question in real-time every 80 ms:

Has the user finished speaking? → ST (start talking — agent responds) or CL (continue listening — user is mid-sentence)

---

Models

File	Size	Description
best.pt	534 MB	Dualturn transformer backbone — predicts VAD/EOT/BOT per channel
endpoint_clf.pkl	370 KB	Logistic regression endpoint classifier — P(ST) from 10 signals

---

How it works

Dual-channel audio (24 kHz stereo)
      │
      ▼  every 80 ms  (Mimi encoder → dualturn transformer)
Per-frame signals:
  vad_user, vad_agent          ← is each speaker currently talking?
  eot_user, eot_agent          ← near end of their turn?
  bot_user, bot_agent          ← beginning a new turn?
  fvad_user_short/long         ← fast VAD (Silero), two smoothing windows
  fvad_agent_short/long
      │
      ▼  VAD edge detector  (watches vad_user crossing 0.5)
      │
      └── VAD offset detected (user stopped) + agent silent?
              │
              ▼  endpoint_clf.predict_proba([10 signal values])
              P(ST) >= 0.30  →  ST  ✓  agent should respond
              P(ST) <  0.30  →  CL     wait, user paused mid-sentence

Threshold = 0.30 — tuned on held-out test set to maximise ST recall (99% recall, 90% precision on test set).

---

Install

pip install torch torchaudio joblib scikit-learn huggingface_hub silero-vad

---

Usage

From HuggingFace (recommended)

from endpointing import DualTurnEndpointing

model = DualTurnEndpointing.from_pretrained("anyreach/dualturn-endpointing")

# Offline: process a stereo WAV file
frames, endpoints = model.process_file(
    "call.wav",
    user_channel=0,    # which stereo channel is the user
    agent_channel=1,
)

for ep in endpoints:
    print(f"t={ep['t_s']:.2f}s  action={ep['action']}  P(ST)={ep['p_st']:.3f}")

From local files

model = DualTurnEndpointing(
    backbone_path   = "best.pt",
    classifier_path = "endpoint_clf.pkl",
    device          = "cuda",
)

Streaming (real-time, 80 ms chunks)

model  = DualTurnEndpointing.from_pretrained("anyreach/dualturn-endpointing")
stream = model.stream(user_channel=0, agent_channel=1)

for chunk in audio_source():          # chunk: np.ndarray (2, 1920) float32
    result = stream.push(chunk)
    if result:
        if result["action"] == "ST":
            agent.start_responding()
        else:
            agent.keep_waiting()

CLI

# From HuggingFace
python endpointing.py --audio call.wav --from-hf anyreach/dualturn-endpointing

# From local files
python endpointing.py \
    --audio call.wav \
    --backbone best.pt \
    --classifier endpoint_clf.pkl \
    --user-channel 0 \
    --agent-channel 1 \
    --out-json results.json

---

Output format

Each endpoint decision:

{
  "t_s":    9.70,
  "action": "ST",
  "p_st":   0.984,
  "signals": {
    "vad_user": 0.02, "vad_agent": 0.01,
    "eot_user": 0.91, "eot_agent": 0.03,
    "bot_user": 0.01, "bot_agent": 0.05,
    "fvad_user_short": 0.03, "fvad_user_long": 0.12,
    "fvad_agent_short": 0.01, "fvad_agent_long": 0.02
  }
}

---

Performance (held-out test set, 15 calls)

Metric	Value
ST recall	99%
ST precision	90%
CL recall	6%
AUC	0.853
Threshold	0.30

The model is tuned for high ST recall — it almost never misses a real turn end. CL recall is low by design (we'd rather respond slightly early than make the user repeat themselves).