---
language:
- kk
- ru
license: apache-2.0
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
- whisper
- speech-recognition
- kazakh
- asr
- fine-tuned
base_model: openai/whisper-large-v3
datasets:
- ISSAI/KSC
- google/fleurs
model-index:
- name: whisper-large-v3-turbo-kk
  results:
  - task:
      type: automatic-speech-recognition
      name: Speech Recognition
    dataset:
      name: ISSAI KSC (test)
      type: ISSAI/KSC
      split: test
    metrics:
    - type: wer
      value: 10.62
      name: WER
---

#  whisper-large-v3-tulpar

A fine-tuned [Whisper Large V3](https://huggingface.co/openai/whisper-large-v3) model optimized for **Kazakh** (қазақ тілі) speech recognition.

> *"Жігіт - ісімен, ат - тұлпарымен."*
> *(A man is known by his deeds, a horse — by its spirit.)*

We got tired of Whisper confusing Kazakh with Turkish and hallucinating random text, so we fixed it. 900+ hours of Kazakh speech, 4x H200 GPUs, and a lot of tea later — here we are.

## What's this?

Full fine-tune of all 1.55B parameters of `openai/whisper-large-v3` on ~900 hours of Kazakh speech data. Not LoRA, not adapters — the whole thing, because we had the GPU memory and the ambition.

**Current WER: 10.62%** on ISSAI KSC test set (down from ~20%+ with vanilla Whisper).

This is v2. We're not done yet.

## Quick Start

```python
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

# Load
processor = WhisperProcessor.from_pretrained("olzhasAl/whisper-large-v3-turbo-kk")
model = WhisperForConditionalGeneration.from_pretrained("olzhasAl/whisper-large-v3-turbo-kk")
model.eval()

# Transcribe
audio, sr = librosa.load("kazakh_audio.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
forced_ids = processor.get_decoder_prompt_ids(language="kazakh", task="transcribe")

generated = model.generate(
    inputs.input_features,
    forced_decoder_ids=forced_ids,
    max_new_tokens=256,
    num_beams=5,
    no_repeat_ngram_size=4,
)

text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)
```

## Training Details

| Parameter | Value |
|---|---|
| Base model | `openai/whisper-large-v3` |
| Parameters | 1.55B (full fine-tune) |
| Training data | ~900 hours Kazakh speech |
| GPUs | 4x NVIDIA H200 (143GB HBM3e each) |
| Precision | BF16 mixed precision |
| Optimizer | AdamW (lr=1e-5, cosine schedule) |
| Batch size | 256 effective (32/GPU × 4 GPUs × 2 accum) |
| Epochs | 1 |

### Data Sources

| Dataset | Hours | Type |
|---|---|---|
| ISSAI KSC | ~335h | Studio recordings |
| farabi-lab | ~554h | Diverse speech |
| FLEURS kk_kz | ~12h | Standard phrases |
| YouTube (curated) | ~57h | News, interviews, lectures |

Total: **~957 hours** of verified Kazakh speech.

### Preprocessing

- Audio: 16kHz mono WAV
- VAD: Silero VAD (float32, threshold=0.5)
- Segments: 5-30 seconds
- Language filtering: Kazakh-specific character detection (ә, і, ң, ғ, ү, ұ, қ, ө, һ)

## Evaluation

| Benchmark | WER |
|---|---|
| ISSAI KSC test | **10.62%** |
| FLEURS kk_kz test | TBD |

*Target: < 8% (v3), < 5% (v4+)*

## Tips & Tricks

**For best results:**
- Use `num_beams=5` and `no_repeat_ngram_size=4` — prevents repetition loops
- Set `language="kazakh"` explicitly — auto-detect may confuse KK with Turkish
- For mixed KK/RU audio: use this model for Kazakh segments + original Whisper for Russian segments

**Known quirks:**
- Language detection head is biased toward Kazakh (trained only on KK data)
- May hallucinate on pure Russian input — don't force `language="kazakh"` on Russian speech
- Code-switching (KZ↔RU within one sentence) is partially supported but not perfect yet

## What's Next

- v3: Training on ~3000+ hours (more YouTube data, podcasts)
- Sub-8% WER target
- Better code-switching support
- KenLM integration for post-processing

## Citation

```bibtex
@misc{whisper-large-v3-turbo-kk,
  author = {Olzhas Alseitov},
  title = {whisper-large-v3-turbo-kk: Fine-tuned Whisper for Kazakh Speech Recognition},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/olzhasAl/whisper-large-v3-turbo-kk}
}
```

## License

Apache 2.0

---