---
tags:
- speech
- gender-recognition
- paralinguistics
- audio-classification
- conformer
- lora
- ecapa-tdnn
license: other
license_name: meralion-public-license
license_link: >-
  https://huggingface.co/q-wang/MERaLiON-GI-v1/blob/main/MERaLiON-Public-Licence-v3.pdf
pipeline_tag: audio-classification
---
# MERaLiON-GR: Multi-lingual Speech Gender Recognition system that performs bi-nary classification (female / male) on English and Southeast Asian (SEA) languages.



- 🤗 Model: [MERaLiON/MERaLiON-GI-v1](https://huggingface.co/MERaLiON/MERaLiON-GI-v1)
- 🎤 Live demo: [meralion.org/analysis](https://meralion.org/analysis)


---
## 🧠 Architecture Overview

- **Modality:** Speech-only model
- **Task:** Audio Classification (Gender Recognition)
- **Input Sampling Rate:** 16,000 Hz
- **Output Classes:** Female (0), Male (1)

| Component | Description |
|---|---|
| **Backbone** | MERaLiON-SpeechEncoder-2 — 24-layer Conformer transformer, pre-trained with a Best-RQ objective |
| **Adaptation** | LoRA adapters in the attention projection layers (rank = 16, scaling factor = 16, rsLoRA normalization, dropout = 0.05) |
| **Layer aggregation** | Learned soft-attention over all 25 hidden-state outputs (input embeddings + 24 layer outputs) |
| **Downstream network** | ECAPA-TDNN — 3 SE-Res2Net blocks (dilations 1, 2, 3; SE reduction factor 8); BatchNorm replaced with GroupNorm |
| **Pooling** | Attention pooling → fixed-length speaker embedding, projected to 256-d |
| **Classification head** | Linear head with RMSNorm + GELU activation → 2-class output (Female / Male) |

The design achieves parameter-efficient adaptation for multilingual, paralinguistic modeling.

More details on model architecture, training, and evaluation are available in the technical report: [arxiv.org/abs/2608.04433](https://arxiv.org/abs/2608.04433)

---

## 📊 Performance Overview
### Public Benchmarks (Accuracy %)

| Language | Test Set | Vox-Profile | **MERaLiON-GR** | Audio-LLM |
|---|---|---|---|---|
| English | FLEURS | 99.69 | **100.00** | 49.61 |
| English | IEMOCAP | 97.31 | **98.90** | 97.21 |
| English | Common Voice | 92.60 | **93.90** | 52.10 |
| Chinese | Common Voice | 96.10 | **98.10** | 64.80 |
| Malay | SMALDUSC | **97.60** | 93.20 | 57.10 |
| Tamil | OpenSLR | 98.30 | **100.00** | 53.70 |
| Tamil | EmoTa | 94.44 | **99.15** | 69.98 |
| Tamil | Common Voice | 92.30 | **94.00** | 51.60 |
| Thai | THAI-SER | **89.32** | 87.23 | 79.05 |
| Thai | Thai Elderly | 96.57 | **100.00** | 72.18 |
| Thai | Common Voice | 96.52 | **97.86** | 59.97 |
| Vietnamese | Common Voice | 96.08 | **99.22** | 36.08 |
| Indonesian | IndoWave | 95.33 | **98.33** | 71.00 |
| Indonesian | Common Voice | 95.00 | **97.40** | 50.02 |
| Khmer | FLEURS | 99.74 | 99.74 | 69.80 |

MERaLiON-GR outperforms Vox-Profile on 12 of 15 public test sets and matches it on 1 set. 

### Singapore Language Performance

| Language | Test Set | Vox-Profile | **MERaLiON-GR** |
|---|---|---|---|
| Chinese | SG-ECMT-Chinese | 89.25 | **91.85** |
| Singlish | SG-ECMT-Singlish | 88.45 | **92.10** |
| Malay | SG-ECMT-Malay | 86.56 | **90.88** |
| Tamil | SG-ECMT-Tamil | 90.57 | **94.73** |

In-the-wild 10–30 second Singapore speech segments (evaluated with a 2-second sliding window), MERaLiON-GR consistently outperforms Vox-Profile, with gains up to **+4.32 pp** (Malay) and **+4.16 pp** (Tamil).

## Baselines

- **Vox-Profile**: https://huggingface.co/tiantiaf/wavlm-large-age-sex state-of-the-art standalone speech foundation model for gender recognition
- **MERaLiON-v2** ([MERaLiON-2-10B](https://huggingface.co/MERaLiON/MERaLiON-2-10B)): general-purpose Audio-LLM, prompted with a fixed instruction to output a gender label

---
## ⚙️ Usage Examples

```python
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoFeatureExtractor

# 1. Load the model and feature extractor
model_id = "MERaLiON/MERaLiON-GR-v1"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device)
model.eval()

# 2. Make sure your waveformes are 16000 sampling rate
wav, sr = sf.read("sample.wav")
if wav.ndim > 1: wav = wav.mean(axis=1)
if sr != 16000:
    wav = torchaudio.functional.resample(torch.tensor(wav).unsqueeze(0), sr, 16000).squeeze(0).numpy()

inputs = feature_extractor( 
        [wav], sampling_rate=16000, return_tensors="pt", padding=True
    )
inputs = {k: v.to(device) for k, v in inputs.items()}

# inference. The model returns (hidden_states, logits)
with torch.no_grad():
    _, logits = model(**inputs)
        
probs = F.softmax(logits, dim=-1)
pred_class = torch.argmax(probs, dim=-1).item()
    
labels_map = {0: "female", 1: "male"}
print("Predicted gender: {labels_map[pred_class]}")
```

## Citation

If you use this model, please cite the corresponding technical report:

```bibtex
@article{meralion2026gr,
  title={MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages},
  author={{MERaLiON Team}},
  journal={arXiv preprint arXiv:2608.04433},
  year={2026}
}
```

Related work referenced in this report:

```bibtex
@inproceedings{wang2025benchmarking,
  title={Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data},
  author={Wang, Qiongqiong and Sailor, Hardik Bhupendra and Liu, Tianchi and Zhang, Wenyu and Huzaifah, Muhammad and Lertcheva, Nattadaporn and Sun, Shuo and Chen, Nancy F and Wu, Jinyang and Aw, AiTi},
  booktitle={Findings of EMNLP 2025},
  year={2025}
}

@inproceedings{cpqa_interspeech,
  title={Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken {QA} Generation},
  author={Wang, Qiongqiong and Sailor, Hardik B and Liu, Tianchi and Aw, Ai Ti},
  booktitle={Proc. Interspeech},
  year={2025}
}

@inproceedings{cpqa_asru,
  title={Incorporating Contextual Paralinguistic Understanding in Large Speech-Language Models},
  author={Wang, Qiongqiong and Sailor, Hardik B and Wong, Jeremy H. M. and Liu, Tianchi and Sun, Shuo and Zhang, Wenyu and Huzaifah, Muhammad and Chen, Nancy and Aw, Ai Ti},
  booktitle={Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year={2025}
}
```