README.md

---
language:
- bho
- hi
license: cc-by-4.0
library_name: transformers
base_model: openai/whisper-small
pipeline_tag: automatic-speech-recognition
tags:
- whisper
- automatic-speech-recognition
- bhojpuri
- indic
- low-resource
datasets:
- ARTPARK-IISc/Vaani-transcription-part
metrics:
- wer
- cer
model-index:
- name: whisper-small-bhojpuri-merged
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      name: Vaani (Bhojpuri)
      type: ARTPARK-IISc/Vaani-transcription-part
      config: Bhojpuri
      split: test
    metrics:
    - type: wer
      value: 36.41
      name: WER
    - type: cer
      value: 17.05
      name: CER
---

# Whisper-small Bhojpuri (merged)

`openai/whisper-small` with a Bhojpuri LoRA adapter already merged in. Standalone
weights — no PEFT dependency, no adapter loading step.

**The LoRA adapter, full training details, evaluation methodology, and the
leakage audit live in the adapter repo:
[Aditya109/whisper-small-bhojpuri-lora](https://huggingface.co/Aditya109/whisper-small-bhojpuri-lora).**
Read that one if you want to know how these numbers were produced.

## Results

Official Vaani Bhojpuri test split, n = 1,426. Every row measured on the same
utterances, with the same normalizer and the same metric implementation.

| Model | Params | WER ↓ | CER ↓ |
|---|---|---|---|
| `openai/whisper-small` (untrained) | 244M | 122.07 | 78.75 |
| `openai/whisper-large-v3` (untrained) | 1.55B | 65.93 | 37.68 |
| `ARTPARK-IISc/SraVaani` | 430M | **34.80** | 20.29 |
| **This model** | 244M | 36.41 | **17.05** |

Against SraVaani — the model trained by the dataset's own authors — this is a
split decision, and both halves are statistically significant under a paired
bootstrap: SraVaani is better on WER by 1.61 (95% CI [+0.83, +2.44]), this
model is better on CER by 3.24 (95% CI [−3.87, −2.53]).

## Usage

```python
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

REPO = "Aditya109/whisper-small-bhojpuri-merged"

processor = WhisperProcessor.from_pretrained(REPO, language="hi", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, attn_implementation="sdpa").to("cuda").eval()

feats = processor.feature_extractor(
    audio_16k, sampling_rate=16_000, return_tensors="pt"
).input_features.to("cuda", dtype=torch.bfloat16)

ids = model.generate(feats, max_new_tokens=200, num_beams=1)
print(processor.batch_decode(ids, skip_special_tokens=True)[0])
```

Weights are stored in float32 so you can cast to whatever your hardware wants.

### Notes

- **Audio must be 16 kHz mono.**
- **The language token is `hi`.** Whisper has no Bhojpuri token; Hindi is the
  nearest proxy, and the dialect knowledge is in the fine-tuned weights. The
  decoding defaults are baked into `generation_config`, so it transcribes
  Bhojpuri out of the box rather than language-detecting into Hindi.
- Being a plain Whisper checkpoint, this converts for **faster-whisper**,
  **whisper.cpp** and **WhisperX**. That portability is the main argument for
  it over a NeMo Conformer.
- Evaluated with greedy decoding on utterance-level clips. Long-form audio and
  beam search are untested.

## Data and limitations

Trained on `ARTPARK-IISc/Vaani-transcription-part` (config `Bhojpuri`),
~24 hours of spontaneous image-description speech, using the dataset's official
splits unmodified.

**Vaani has no speaker ID column**, so speaker-disjointness is trusted from the
official splits, not verified — by me or by anyone else working from the public
dataset. Text-level leakage was audited and ruled out; details and the audit
script are in the adapter repo.

Other limitations: no standard-Hindi regression check was run, so the model may
have degraded on standard Hindi; domain coverage is limited to one collection
protocol; and some of the residual error is annotation noise in the references
rather than model error.

## Licensing

- Base model `openai/whisper-small` is **MIT**.
- Training data (Vaani) is **CC-BY-4.0**, which requires attribution, so this
  model is released under **CC-BY-4.0**. No Vaani audio or transcripts are
  redistributed here.

Attribution for the dataset, as requested on its
[dataset page](https://huggingface.co/datasets/ARTPARK-IISc/Vaani-transcription-part):

```bibtex
@misc{pulikodan2026vaanicapturinglanguagelandscape,
      title={VAANI: Capturing the language landscape for an inclusive digital India},
      author={Sujith Pulikodan and Abhayjeet Singh and Agneedh Basu and Nihar Desai and Pavan Kumar J and Pranav D Bhat and Raghu Dharmaraju and Ritika Gupta and Sathvik Udupa and Saurabh Kumar and Sumit Sharma and Vaibhav Vishwakarma and Visruth Sanka and Dinesh Tewari and Harsh Dhand and Amrita Kamat and Sukhwinder Singh and Shikhar Vashishth and Partha Talukdar and Raj Acharya and Prasanta Kumar Ghosh},
      year={2026},
      eprint={2603.28714},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2603.28714}
}
```