---
title: Whisper Speaker Diarization
emoji: 🎎
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 5.16.0
python_version: "3.10"
app_file: app.py
pinned: false
tags:
- whisper-event
---

Whisper speaker diarization on T4 using `faster-whisper` (large-v3, fp16) and `pyannote.audio` 3.1. Outputs CSV + SRT.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

## UI Highlights

- **Two-pane brutalist workspace** — 4-step stepper (Source · Configure · Process · Review) on the left, tabbed results dock (Transcript · Audio · Downloads · Diagnostics) on the right.
- **Stage-aware progress** — single PROCESS button auto-chains YouTube download → convert → transcribe → diarize → align → format, with a live monospace log card (first feedback under 2s).
- **Interactive transcript** — per-speaker color chips with a deterministic palette, click-row-to-seek audio playback, and editable speaker rename that rewrites CSV + SRT in place without re-running the pipeline.
- **Inline error card** — known failure modes (missing HF_TOKEN, GPU OOM, YouTube auth) render an actionable remediation card next to the progress log.

## Required Setup

This Space depends on the gated `pyannote/speaker-diarization-3.1` model. Before the app can run you must:

1. Visit https://huggingface.co/pyannote/speaker-diarization-3.1 and accept the user agreement (also accept the `pyannote/segmentation-3.0` agreement linked there).
2. Create a Hugging Face access token at https://huggingface.co/settings/tokens with `read` scope.
3. Add the token as a private Space secret named `HF_TOKEN`. Locally, export `HF_TOKEN=...` or place it in `.env`.

Without `HF_TOKEN` and the accepted agreements, diarization will fail with a clear runtime error.

## Persistent Cache

Model weights (~3 GB Whisper + ~2 GB pyannote) are cached to `MODEL_CACHE_DIR`. Resolution order:

- `MODEL_CACHE_DIR` env var, if set (recommended: a Space persistent storage path such as `/data/models`)
- Otherwise falls back to `output/cache/models` inside the Space (ephemeral; re-downloaded on cold start)

When the cache dir is resolved the app sets `HF_HOME`, `HUGGINGFACE_HUB_CACHE`, and `TORCH_HOME` to the same location so all framework caches share it.

## YouTube Downloads On Hugging Face Spaces

Some YouTube videos require authenticated cookies before `yt-dlp` can download them. Browser-cookie auth and YouTube username/password login do not work in Hugging Face Spaces because the container does not have your local browser profile and YouTube blocks password login for `yt-dlp`.

To support those videos, export fresh YouTube cookies in Netscape `cookies.txt` format and add them as a private Space secret named `YTDLP_COOKIES_CONTENT`, then restart the Space. Locally, you can also place the exported file at `cookies.txt` or set `YTDLP_COOKIES_FILE` to another path.

Downloaded YouTube videos in `output/youtube/` are cleaned up automatically after 7 days (files newer than 1 hour are skipped to avoid race conditions with in-flight jobs).