---
language:
  - luo
  - kln
  - kik
  - mas
  - som
  - sw
license: cc-by-4.0
tags:
  - audio
  - automatic-speech-recognition
  - w2v-bert-2.0
  - ctc
  - ctranslate2
  - int8
  - kenlm
  - african-languages
base_model: leophill/w2v-bert-2.0-afrivoices-ea-6l-asr
base_model_relation: quantized
pipeline_tag: automatic-speech-recognition
metrics:
  - wer
  - cer
library_name: ctranslate2
datasets:
- DigitalUmuganda/Afrivoice_Swahili
- DigitalUmuganda/Afrivoice
- Anv-ke/kikuyu
- Anv-ke/Kalenjin
- Anv-ke/Somali
- Anv-ke/Dholuo
- Anv-ke/Maasai
---

# Wav2Vec2-BERT 2.0 ASR for 6 East African Languages - Edge (CTranslate2 & ONNX int8)

Edge conversions of
[leophill/w2v-bert-2.0-afrivoices-ea-6l-asr](https://huggingface.co/leophill/w2v-bert-2.0-afrivoices-ea-6l-asr) : 
**CTranslate2 int8** (weight mapping — CTranslate2 implements the
Wav2Vec2-BERT architecture natively) and **ONNX int8** (graph export with the
CTC head baked in, dynamically quantised).

**CTranslate2** is the recommended
engine (fastest, smallest RSS); ONNX is provided for runtimes where
CTranslate2 is unavailable (onnxruntime is broadly portable, incl. mobile).

One 580M-parameter model transcribes all six
languages — Dholuo (luo), Kalenjin (kln), Kikuyu (kik), Maasai (mas), Somali
(som), Swahili (swa) — over a shared 34-character union vocabulary (Kikuyu
ĩ/ũ preserved). No language token or language input exists: the model
transcribes whichever of the six languages it hears.

Built for the **AfriVoices East Africa ASR Hackathon** (Kaggle; Digital
Umuganda & Maseno Center for Applied Artificial Intelligence) — 2nd place (provisional),
macro-WER 0.34819 — and its edge constraints: <1B parameters, CPU inference
within 8 GB RAM, near-real-time transcription.

## ⚠ CTranslate2 ≥ 4.8 output contract

Recent CTranslate2 releases (measured at 4.8.1) return **encoder hidden
states `(T, 1024)`** from `encode()` — not CTC logits. This repository ships
**`lm_head.npz`** (the 1024→34 CTC projection, fp32, ~140 KB); apply it after
`encode()` as shown below. Older CTranslate2 builds that emit `(T, 34)`
logits directly need no projection.

## Usage — CTranslate2 (recommended, CPU, torch-free)

```python
import re, numpy as np, soundfile as sf, ctranslate2
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer

repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge"   # this repo
model = ctranslate2.models.Wav2Vec2Bert(
    repo, device="cpu", compute_type="int8", intra_threads=4)
fe  = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)

audio_file = "audio.wav"
audio, sr = sf.read(audio_file)          # 16 kHz mono
feats = fe(audio, sampling_rate=sr, return_tensors="np").input_features
out = np.asarray(model.encode(
    ctranslate2.StorageView.from_array(feats.astype(np.float32)),
    to_cpu=True))[0]

if out.shape[-1] != 34:                   # CT2 >= 4.8: hidden states
    z = np.load("lm_head.npz")
    out = out @ z["weight"].T + z["bias"]

text = tok.decode(out.argmax(-1), group_tokens=True)   # greedy CTC
text = re.sub(" +", " ", text).strip()   # clean extra spaces
print(text)
```

## Usage — ONNX Runtime (CPU, torch-free)

```python
import re, numpy as np, soundfile as sf, onnxruntime as ort
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer
from huggingface_hub import hf_hub_download

repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge"      # this repo
onnx_path = hf_hub_download(repo, "onnx/model_int8.onnx")     # cached locally
sess = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
fe  = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)

audio_file = "audio.wav"
audio, sr = sf.read(audio_file)          # 16 kHz mono

enc = fe(audio, sampling_rate=sr, return_tensors="np")
mask = enc.get("attention_mask",
               np.ones(enc.input_features.shape[:2], dtype=np.int64))
logits = sess.run(["logits"], {
    "input_features": enc.input_features.astype(np.float32),
    "attention_mask": np.asarray(mask, dtype=np.int64),
})[0][0]                                   # (T, 34) — head is in the graph
# (quantisation covers MatMul/Gemm only — no ConvInteger nodes)

text = tok.decode(logits.argmax(-1), group_tokens=True)   # greedy CTC
text = re.sub(" +", " ", text).strip()   # clean extra spaces
print(text)
```

For best accuracy with either engine, decode with **pyctcdecode + a per-language 5-gram KenLM**
(beam width 100; the hackathon submission used per-language α ∈ {0.25, 0.5},
β = 0.0, Swahili greedy) and window long audio at **28 s with 4 s overlap**.
The complete decode pipeline (windowing, stitching, LM configs) is in the
solution repository.

## Edge benchmark

Conditions: this checkpoint, identical clips, identical pyctcdecode + KenLM
decode path, x86 CPU, Somali 200-clip long-form-skewed sample, **4 threads**:

| engine | ΔWER vs fp32 | RTF total | peak RSS |
|---|---:|---:|---:|
| torch fp32 (reference) | — | 5.9 | 5.1 GB |
| **CTranslate2 int8** | **−0.003** | **0.412** | **2.7 GB** |
| ONNX Runtime int8 | −0.003 | 0.449 | 4.7 GB |

int8 costs no measurable WER (Δ within beam tie-breaking noise, two
independent engines agreeing).

Full-test-set validation (4 threads per process — one edge device's
budget): per-language RTF 0.51–0.64 (worst: Kalenjin, largest-lexicon LM);
worst-case peak RSS **6.2 GB** (Somali, longest clips) using sequential
shard recycling to bound allocator growth — within the 8 GB envelope.
Measured on a Galaxy S21: RTF 1.308 (Somali sample).

## Intended use & limitations

16 kHz mono speech in the six covered languages; performance varies strongly
by register (scripted read speech vs long-form spontaneous) and follows the
training-data conventions of the hackathon corpora. Not a language
identifier; audio outside the six languages produces unusable text.

## Citation

```bibtex
@misc{w2v-bert-2.0-afrivoices-ea-6l-asr,
  author = {Leopold Hillah},
  title = {Jointly Finetuning Wav2Vec-BERT 2.0 for 6 East African Languages},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/leophill/w2v-bert-2.0-afrivoices-ea-6l-asr}
}
@misc{afri-voices-east-africa-asr-hackathon,
  author = {{Digital Umuganda} and {Maseno University} and
            {Maseno Center for Applied Artificial Intelligence}},
  title = {AfriVoices East Africa: ASR Hackathon},
  year = {2026},
  howpublished = {\url{https://kaggle.com/competitions/afri-voices-east-africa-asr-hackathon}},
  note = {Kaggle}
}
```