---
language:
- la
license: mit
base_model: facebook/wav2vec2-xls-r-300m
tags:
- automatic-speech-recognition
- wav2vec2
- latin
- classical-latin
- macrons
- audio
- onnx
- optimum
- int8
- gradio
datasets:
- njand/llpsi-speech-dataset
metrics:
- wer
- cer
pipeline_tag: automatic-speech-recognition
model-index:
- name: Wav2Vec2 XLS-R 300M - Latin ASR
  results:
  - task:
      type: automatic-speech-recognition
      name: Speech Recognition
    dataset:
      type: njand/llpsi-speech-dataset
      name: LLPSI Speech Dataset
    metrics:
    - name: 5-Fold CV Strict WER
      type: wer
      value: 27.01
    - name: 5-Fold CV Strict CER
      type: cer
      value: 12.95
    - name: 5-Fold CV Normalized WER
      type: wer
      value: 25.26
    - name: 5-Fold CV Normalized CER
      type: cer
      value: 12.37
---

# Wav2Vec2 XLS-R 300M - Latin ASR (Restored Classical Pronunciation)

Fine-tuned version of [`facebook/wav2vec2-xls-r-300m`](https://huggingface.co/facebook/wav2vec2-xls-r-300m) for Latin Automatic Speech Recognition (ASR), trained on reading passages from `njand/llpsi-speech-dataset`.

- **Live Demo:** [Gradio Interface](https://huggingface.co/spaces/njand/wav2vec2-xls-r-latin)
- **Source Code:** [GitHub Repository](https://github.com/njand/latin-asr)
- **Base Model:** [`facebook/wav2vec2-xls-r-300m`](https://huggingface.co/facebook/wav2vec2-xls-r-300m)
- **Dataset:** [`njand/llpsi-speech-dataset`](https://huggingface.co/datasets/njand/llpsi-speech-dataset) (currently private)
- **Pronunciation Standard:** Restored Classical Pronunciation
- **Orthography:** Macrons (ā, ē, ī, ō, ū, ȳ); consonantal j/v

---

## 🛠️ Model Variants & Optimization

To facilitate production deployment on CPU-based infrastructure, this repository provides the model in three formats:

| Format | Precision | File Size | Recommended Use Case |
| :--- | :--- | :--- | :--- |
| **PyTorch** | FP32 | 1.26 GB | Training, fine-tuning, and research |
| **ONNX** | FP32 | 1.26 GB | Cross-platform inference |
| **ONNX Quantized** | INT8 | **355 MB** | **Production, Edge, & CPU Inference** |

> **Why Quantized?** The INT8 version offers ~3x faster inference speed on standard CPUs compared to the original FP32 PyTorch weights while retaining near-identical transcription accuracy.

---

## 🚀 Quickstart

### Option 1: Standard Inference (PyTorch)
```python
from transformers import pipeline

transcriber = pipeline("automatic-speech-recognition", model="njand/wav2vec2-xls-r-latin")
result = transcriber("sample.wav")
print(result["text"])

```

### Option 2: Optimized Inference (Optimum ONNX Runtime)

For CPU inference, I recommend using the INT8 quantized model. Install dependencies: `pip install "optimum[onnxruntime]"`

```python
from optimum.onnxruntime import ORTModelForCTC
from transformers import AutoProcessor

# Load the quantized model
model = ORTModelForCTC.from_pretrained(
    "njand/wav2vec2-xls-r-latin", 
    file_name="model_quantized.onnx"
)
processor = AutoProcessor.from_pretrained("njand/wav2vec2-xls-r-latin")

# Inference pipeline
inputs = processor("sample.wav", return_tensors="pt")
outputs = model(**inputs)
# ... decode outputs

```

---

## 🔤 Orthography & Text Normalization

This model transcribes audio using the following orthographic conventions:

* **Vowel Quantity:** Macronizes long vowels (ā, ē, ī, ō, ū, ȳ).
* **Consonantal vs. Vocalic Glides:** Distinguishes consonantal **j** and **v** from vocalic **i** and **u** (e.g., *ējiciō* rather than *eicio*, *vīvus* rather than *uiuus*).
* **Casing & Punctuation:** Transcribes lowercase text with no punctuation.

---

## 📊 Cross-Validation Performance

Evaluated across 5-fold cross-validation on `njand/llpsi-speech-dataset`:

| Fold | Strict WER | Strict CER | Norm WER | Norm CER |
| :--- | :---: | :---: | :---: | :---: |
| **Fold 0** | 27.09% | 13.06% | 25.26% | 12.48% |
| **Fold 1** | 26.33% | 12.92% | 24.61% | 12.33% |
| **Fold 2** | 26.27% | 12.74% | 24.59% | 12.18% |
| **Fold 3** | 27.21% | 12.91% | 25.26% | 12.27% |
| **Fold 4** | 28.16% | 13.12% | 26.59% | 12.58% |
| **Average (CV)** | **27.01%** | **12.95%** | **25.26%** | **12.37%** |

> **Note:** Normalized WER/CER measure core word recognition (macrons stripped, j/v → i/u), while Strict WER/CER enforce exact macron placement and **j/v** orthography.

---

## ⚡ Environmental Impact & Compute

- **Hardware:** NVIDIA L4 GPU (24GB VRAM) via Modal
- **Total Training Time:** 5.87 hours
- **Estimated Carbon Emissions:** 0.3135 kg CO2eq

---

## ⚠️ Limitations & Out-of-Scope Use

* **Pronunciation Variances:** Performance will drop on audio using **Ecclesiastical / Italianate** pronunciation (e.g., pronouncing *c* before *e/i* as `/tʃ/` rather than `/k/`).
* **Audio Conditions:** Optimized for clear, single-speaker reading. Background noise or overlapping speakers will reduce accuracy significantly.