voice-gender-classifier-onnx-q8-v2

ONNX export with dynamic q8 (UInt8, weight-only, per-channel) quantization of JaesungHuh/voice-gender-classifier, prepared for Transformers.js inference in browser and mobile contexts.

v2 of voice-gender-classifier-onnx-q8. Same upstream weights, same export, same file layout, same size class — one change to the quantization recipe that recovers full FP32 accuracy (see next section). v1 remains available for reproducibility of results published against it.

Used in Syrinx, an open-source voice-training tool. MIT licensed — anyone is welcome to use this for any purpose under MIT terms, not Syrinx-specific.

What changed in v2

v1 quantized every eligible node. A per-node sensitivity analysis (quantize one node at a time, measure mean |Δp| against FP32 on held-out audio windows) showed the quantization error was almost entirely concentrated in one node: the attentive-statistics-pooling matmul, which multiplies two runtime activations (attention weights × frame features). Dynamic quantization quantizes both operands on the fly, and that product took ~10× more damage (mean |Δp| 0.155) than the worst of the 38 convolutions (0.016) — the convolutions, whose weights are static and per-channel calibrated, were essentially unharmed.

v2 uses the identical recipe with that single node excluded from quantization (nodes_to_exclude=["node_matmul"]), so the attention product runs in FP32 while everything else stays int8:

	v1 (all nodes q8)	v2 (matmul excluded)	FP32 reference
Hillenbrand male accuracy	95.6 % (43/45)	100 % (45/45)	100 %
Hillenbrand female accuracy	95.8 % (46/48)	100 % (48/48)	100 %
Within-speaker raw_std, median (m / f)	0.216 / 0.196	0.010 / 0.025	0.013 / 0.019
Browser WASM inference, 0.75 s window (desktop median)	~50 ms	~46 ms	~431 ms
File size	15.8 MB	16.1 MB	62 MB

v2 matches the FP32 model's accuracy and per-window noise on this corpus while keeping q8 size and speed — slightly faster than v1, because the excluded matmul skips its dynamic quantize/dequantize round-trip. Speakers that v1 persistently misclassified (m45, w46 — previously believed to be inherently borderline voices) classify correctly and confidently under v2 and FP32: their "borderline-ness" was quantization noise, not a property of the model or the voices.

Architecture

ECAPA-TDNN with C=1024 channels:

• 1D Conv1d feature extractor (80 → 1024, kernel=5)
• Three Bottle2neck Res2Net blocks with SE attention (1024-wide, scale=8)
• Multi-frame attention pooling (3 × 1024 → 1536, attention over time) — the attention product is FP32 in v2
• BatchNorm + Linear (3072 → 192 → 2)

Input: raw mono float32 audio at 16 kHz, shape (batch, time). Output: pre-softmax logits, shape (batch, 2) over {0: male, 1: female}.

The model includes its own log-mel preprocessing (logtorchfbank) baked into the ONNX graph — STFT (opset 17+), mel filterbank, log, per-frame mean normalization. Set do_normalize: false in the preprocessor config (already set in this repo's preprocessor_config.json) to avoid double-normalizing the input — the wav2vec2-style audio-side normalization that Transformers.js applies by default would distort the model's internal mel-side normalization.

About the wav2vec2 tag on this model card (HF Hub auto-tagging from config.json): the architecture is genuinely ECAPA-TDNN, not wav2vec2. The model_type: "wav2vec2" field is set deliberately so Transformers.js's audio-classification pipeline routes the model through its Wav2Vec2ForSequenceClassification JS class, which is a thin ONNX wrapper that runs the embedded graph regardless of architecture. Transformers.js doesn't have an ecapa-tdnn model_type registered, and an unrecognized model_type would cause pipeline() to fail at load time. Don't "fix" this field — it is load-bearing.

Usage with Transformers.js

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "audio-classification",
  "Alice-Sabrina-Ivy/voice-gender-classifier-onnx-q8-v2",
  { dtype: "q8" }   // WASM backend — do NOT pass device: "webgpu"; see below
);

// audio: Float32Array of mono samples at 16 kHz
const result = await classifier(audio, { sampling_rate: 16000 });
// → [{ label: "female", score: 0.95 }, { label: "male", score: 0.05 }]

Run this model on the WASM backend, not WebGPU. On ORT-web 1.26 / Chrome 150, the WebGPU execution provider creates a session successfully for this quantized graph but then raises WebGPU validation errors at run time (Invalid BindGroupLayout around Concat) and runs 4× slower than WASM (200 ms vs ~46-52 ms per 0.75 s window, desktop). Because session creation succeeds, a try-WebGPU-catch-fallback pattern will not save you — it selects the broken path. Quantized int8 operators are generally a poor fit for the WebGPU EP; this model's Concat-heavy Res2Net topology makes it worse. (The v1 card's browser numbers were measured through the WebGPU path before this was understood.)

Performance

Measured on the Hillenbrand 1995 vowel corpus¹ (93 speakers, 12 vowels per speaker concatenated into ~7 s recordings, rolling 0.75 s window at 150 ms hop, EMA α=0.2):

metric	value
Female accuracy	100 % (48/48 speakers)
Male accuracy	100 % (45/45 speakers)
Within-speaker raw_std (median, female)	0.025
Within-speaker raw_std (median, male)	0.010

Inference time (q8-v2, single 0.75 s window):

runtime	median	p95
Chrome 150 desktop, onnxruntime-web 1.26 WASM	~46 ms	~59 ms
Chrome 150 desktop, onnxruntime-web WebGPU (broken path — do not use)	~200 ms	~225 ms
Node native ORT (onnxruntime-node, accuracy harness)	~11 ms	~14 ms

The Node-native number is what the accuracy harness runs on but is roughly 18× faster than real browser WASM — for browser-deployment ship decisions, use the browser-runtime numbers. Mobile browser WASM has not been measured for v2 yet; ECAPA's previously observed desktop:mobile browser ratio (~2.4×) predicts ~110–130 ms on Pixel-class hardware.

¹ Hillenbrand, J., Getty, L. A., Clark, M. J., & Wheeler, K. (1995). Acoustic characteristics of American English vowels. The Journal of the Acoustical Society of America, 97(5), 3099–3111.

Limitations & biases

• Binary classification only. This model emits only male/female logits and cannot represent non-binary or unspecified gender identity. For voice training tools, this is a feature limitation worth disclosing to users.
• Trained on VoxCeleb2, which has demographic imbalance — overrepresentation of European/American English-language speakers, less coverage of voices outside that distribution. Per JaesungHuh's original model card: "the model may not represent global population diversity." Expect degraded accuracy on voices that fall outside VoxCeleb2's distribution.
• 100 % on Hillenbrand is a small-corpus result, not a claim of perfection: 93 adult US-English speakers reading sustained vowels in clean conditions. Real-world speech, other languages, children's voices, and noisy environments will be harder. The v1 card's note about "mechanically borderline samples" (m45, w46) is retracted for v2 — those errors were quantization damage — but genuinely ambiguous voices will still produce mid-range scores.

Quantization

q8-v2 ONNX is a derivative work — produced via:

1. PyTorch → ONNX export with torch.onnx.export (opset 18, dynamic axes for batch and time dimensions)
2. The upstream logtorchfbank constructs torchaudio.transforms.MelSpectrogram per-forward, which trips torch.export's data-dependent guard. Patched by lifting MelSpectrogram and the preemphasis kernel into pre-instantiated members on the model before tracing — numerically identical to upstream, just instantiated once instead of per-call.
3. Quantization pre-processing via onnxruntime.quantization.shape_inference.quant_pre_process
4. Dynamic quantization via onnxruntime.quantization.quantize_dynamic with QuantType.QUInt8, per_channel=True, reduce_range=False, nodes_to_exclude=["node_matmul"] (the attentive-pooling activation×activation product — the v2 change)

sha256 of onnx/model_quantized.onnx: fdc2dbdcf99b9217977f7472f7d677dd48219c4759ca3f38d0626b600d86c252

Conversion scripts live in the Syrinx repository (export-jaesunghuh-onnx.py on the perceived-voice-jaesunghuh-tdnn-investigation branch); the sensitivity analysis and full before/after measurements are in measurements/gender-model-latency-2026-07-19.md.

Citation

If you use this model, please cite the original ECAPA-TDNN paper:

@inproceedings{desplanques2020ecapa,
  title={ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification},
  author={Desplanques, Brecht and Thienpondt, Jenthe and Demuynck, Kris},
  booktitle={Interspeech 2020},
  pages={3830--3834},
  year={2020},
  doi={10.21437/Interspeech.2020-2650},
  url={https://arxiv.org/abs/2005.07143}
}

And acknowledge JaesungHuh's voice-gender-classifier fine-tune:

• Repo: https://github.com/JaesungHuh/voice-gender-classifier
• Model card: https://huggingface.co/JaesungHuh/voice-gender-classifier

License

MIT, inherited from JaesungHuh's original. The architecture code (in turn derived from TaoRuijie's ECAPA-TDNN) is also MIT licensed. This ONNX-quantized derivative continues under MIT.

MIT License

Copyright (c) 2024 JaesungHuh
Copyright (c) 2024 Tao Ruijie (original ECAPA-TDNN implementation)
Copyright (c) 2026 Alice Sabrina Ivy (ONNX export + q8 quantization)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Original work

• Architecture: TaoRuijie/ECAPA-TDNN (Tao Ruijie, MIT)
• Fine-tune: JaesungHuh/voice-gender-classifier (Jaesung Huh, MIT) — VoxCeleb2 fine-tune for binary gender classification, reports 98.7 % on VoxCeleb1 test split
• ONNX export + q8-v2 quantization: this repo