---
title: SLM Judge Leaderboard
emoji: ⚖️
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 6.14.0
app_file: app.py
pinned: false
license: apache-2.0
---

# SLM Judge Leaderboard

A meta-leaderboard: not a ranking of TTS models, but of the **judges**
themselves — how well each SLM (speech-language-model) judge tracks human
ratings on Hume's TTS evals. Agreement is the Spearman ρ between the judge's
per-generation scores and individual human votes, computed per eval and
averaged; the table shows each TTS category (the Overall mean stays in the
data but isn't rendered), heatmapped on fixed absolute anchors (0 → 0.75 ρ).
Cells flagged `*` had at least one zero-variance eval (ρ undefined, reported
as 0).

Single-board sibling of the Real World VoiceEQ Benchmark Space — same app
machinery, same data schema, same backing dataset.

## Data

Loaded at runtime from the shared Hugging Face dataset — configure two Space
settings:

- Variable `LEADERBOARD_DATASET` → `HumeAI/hume-speech-leaderboard-data`
  (this Space reads `slm_judge_leaderboard.json` from it)
- Secret `HF_TOKEN` → a read token (needed while the dataset is private)

For local development, the app falls back to [`data/`](data/)
(`pip install -r requirements.txt && python app.py`).