---
license: apache-2.0
base_model: openai/whisper-medium
datasets:
- burakaydinofficial/Whispered
language: [en, de, es, fr, ru, tr, cy, ar, ka, th, zh]
pipeline_tag: automatic-speech-recognition
library_name: transformers
tags: [whisper, mla, multi-head-latent-attention, kv-cache-compression, speech-recognition, multilingual]
---

# Whisper-Medium-MLA (11 languages) — 62.5% smaller decode KV-cache

`openai/whisper-medium` (769M) with decoder self-attention converted MHA→MLA
([Whisper-MLA, arXiv:2603.00563](https://arxiv.org/abs/2603.00563)), recovery-fine-tuned on 11
languages of the CC0 [Whispered](https://huggingface.co/datasets/burakaydinofficial/Whispered)
corpus (32k clips/lang). The largest absolute cache savings of the released family — medium's
per-stream decode cache is ~2.7× small's, so the 62.5% cut buys the most bytes here. For the
family's top accuracy tier, see `whisper-large-v3-mla`.

```python
from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-medium-mla-cv11", trust_remote_code=True)  # transformers==4.46.x
```

## Results (CommonVoice-17 test, n=1500/lang; cost = paired vs identically-trained unconverted control)

| Lang | this model (WER/CER %) | conversion cost | sig |
|---|---|---|---|
| en | 8.7 / 4.4 | +0.41 | ✱ |
| de | 9.1 / 3.2 | +0.39 | ✱ |
| es | 6.6 / 2.2 | +0.19 | ns |
| fr | 13.2 / 5.0 | −0.20 | ns |
| ru | 8.4 / 2.4 | +0.34 | ✱ |
| tr | 13.3 / 3.5 | +0.16 | ns |
| cy | 42.2 / 14.5 | +0.73 | ✱ |
| ar | 24.3 / 9.0 | +0.42 | ns |
| th | 19.8 / 6.2 | CER +0.08 | ns |
| zh | 19.8 / 10.7 | CER +0.53 | ✱ |
| **ka** † | 80.6 / 36.8 | **+0.54** | ns |

Median cost ≈ +0.4; not significant on 6/11 languages (zh cost is significant on CER).
† ka: the compression cost collapses to ns at medium, but ka itself stays unusable (80.6 WER;
capacity was and is its problem).

## The honest scaling note (preregistered prediction, half-held)
We predicted medium's cost at ≈+0.2 before training. Measured: ≈+0.4 — **the size-scaling curve
SATURATES** (tiny ≈+1.9 → base ≈+1.0 → small ≈+0.4 → medium ≈+0.4; approximate per-size medians)
rather than decaying to zero. The
practical claim holds (all languages ≤+1.0, mostly ns); the mechanism reading is an irreducible
truncation cost at this rank, not ever-growing redundancy absorption.

**Matched control now published** — verify the conversion cost yourself: [`burakaydinofficial/whisper-medium-cv11`](https://huggingface.co/burakaydinofficial/whisper-medium-cv11) (trained identically, minus the MHA→MLA conversion). Evaluate both with `scripts/validate.py`.

## Limitations
- **Costs are recovery-mix-specific:** do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
- **What the 62.5% is (cache scope):** it is the *decode self-attention* KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream *total* decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
- **Requires `trust_remote_code=True` and `transformers==4.46.x`** (custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2.
- **Language coverage is the 11 recovery languages** — unseen scripts degrade (measured).
- **Consumer-mic read-speech domain** (real environmental noise included; SNR-ladder on the small tier showed cost flat to 0 dB); greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
- **Training:** 15k steps, warmup+cosine, encoder frozen both arms, dev-checkpoint-selected, bf16 (weights released as fp16).

## FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 9.6 · de 9.7 · es 7.3 · fr 11.0 · ru 12.6 · tr 14.4 · cy 41.4 · ar 20.8 · th 20.0 · zh-CN 11.1 · ka 92.4.
(Full JSON: the release repo's eval artifacts.)