---
title: Turkish ASR Leaderboard
emoji: 🎤
colorFrom: red
colorTo: pink
sdk: gradio
sdk_version: 5.12.0
app_file: app.py
pinned: false
license: mit
tags:
  - turkish
  - asr
  - leaderboard
  - speech-recognition
  - evaluation
---

# 🇹🇷 Turkish ASR Leaderboard 🎤

Comprehensive evaluation and ranking of **Automatic Speech Recognition (ASR)** models for the **Turkish language**.

## 🎯 Overview

This leaderboard provides transparent, reproducible benchmarks for Turkish ASR models across multiple test datasets. All models are evaluated on identical hardware and datasets to ensure fair comparison.

## 📊 Test Datasets

All models are evaluated on the following standard Turkish ASR datasets:

1. **CommonVoice 17.0 Turkish**
   - **Samples**: 9,650 (test split)
   - **Type**: Read speech (crowdsourced)
   - **Source**: [Mozilla Common Voice](https://commonvoice.mozilla.org/)

2. **FLEURS Turkish**
   - **Samples**: 647 (test split)
   - **Type**: Read speech
   - **Source**: [Google FLEURS](https://huggingface.co/datasets/google/fleurs)

## 📈 Evaluation Metrics

- **WER (Word Error Rate)**: Lower is better - measures word-level accuracy
- **CER (Character Error Rate)**: Lower is better - measures character-level accuracy
- **Average WER**: Mean WER across all three test sets
- **RTF (Real-Time Factor)**: Processing speed (lower = faster)

## 🚀 Submit Your Model

Want to add your Turkish ASR model to the leaderboard? Follow these steps:

### 1. Test Your Model

Test your model on our standard datasets:
- CommonVoice 17.0 Turkish (test split)
- FLEURS Turkish (test split)

### 2. Calculate Metrics

Use the `jiwer` library to calculate WER and CER:

```python
import jiwer

wer = jiwer.wer(references, predictions)
cer = jiwer.cer(references, predictions)
```

### 3. Submit Your Results

Submit via one of the following channels:
- **GitHub**: Open an issue or PR
- **Hugging Face**: Comment on this space
- **Email**: Contact the maintainer

### Required Information

- Model name and Hugging Face link
- WER/CER scores for each test set
- Model parameter count
- RTF (Real-Time Factor) on GPU
- Description of training data
- License type

## 🔬 Methodology

- **Hardware**: All models tested on NVIDIA L40S GPU
- **Metrics**: Calculated using `jiwer` library
- **Transparency**: No cherry-picking or selective reporting
- **Reproducibility**: Full evaluation scripts available

## 📚 Resources

- **Benchmark Scripts**: [GitHub Repository](https://github.com/y0mur)
- **Model Collection**: [y0mur on Hugging Face](https://huggingface.co/y0mur)
- **Dataset Details**: Links provided in test datasets section

## 🙏 Acknowledgments

- **Hugging Face** for hosting and infrastructure
- **OpenAI** for Whisper models
- **Meta** for Wav2Vec2 and MMS models
- **Mistral AI** for Voxtral base model
- **Turkish NLP community** for datasets and support

## 📫 Contact

- **Maintainer**: [y0mur](https://huggingface.co/y0mur)
- **Last Updated**: December 2025
- **Questions?**: Open an issue or comment on the space!

## 📜 License

This leaderboard and evaluation code is released under the MIT License.

---

<div align="center">
    <strong>🇹🇷 Proudly Turkish | Made with ❤️ for the Turkish NLP community</strong>
</div>