OpenVoiceOS/paraformer-trilingual-zh-yue-en-onnx

Paraformer trilingual ZH / Cantonese / EN ONNX

A SeACo-Paraformer trained on Mandarin, Cantonese (Yue) and English for onnx-asr. This is the strongest available Cantonese model in this collection: it is character identical to native FunASR on every Cantonese and Mandarin test clip, fp32 and int8 alike, and it replaces OpenVoiceOS/whisper-small as the Cantonese default, since that model is a general multilingual Whisper checkpoint with no Cantonese-specific training.

What this is

Paraformer is the Alibaba FunASR offline non-autoregressive recognizer. A SAN-M encoder reads the audio, a CIF predictor decides how many tokens the utterance has, and a single pass decoder emits all of them at once. There is no decoding loop, so one forward pass gives the transcript.

This checkpoint is a SeACo-Paraformer, the hotword-biasing variant of Paraformer. The graph exposes two extra outputs used by the bias pathway (decoder_hidden, pre_acoustic_embeds), but the paraformer model type in onnx-asr only requests logits and token_num by name, so it loads and runs unmodified, with no hotword biasing applied. Its speech / speech_lengths inputs and logits / token_num outputs match the plain Paraformer contract exactly.

The ONNX graphs here are copied byte for byte from the sherpa-onnx export by csukuangfj. Only the side files changed: tokens.txt became vocab.txt, and config.json carries the FunASR frontend statistics from am.mvn.

Usage

The paraformer model type is on the feat/paraformer branch of the TigreGotico onnx-asr fork.

pip install "onnx-asr[cpu,hub] @ git+https://github.com/TigreGotico/onnx-asr@feat/paraformer"

import onnx_asr

model = onnx_asr.load_model("OpenVoiceOS/paraformer-trilingual-zh-yue-en-onnx")
print(model.recognize("audio.wav"))

Graph contract

Item	Value
Input	speech, float32, [batch, num_frames, 560]
Input	speech_lengths, int32, [batch]
Output	logits, float32, [batch, num_tokens, 11666]
Output	token_num, int32, [batch], the CIF token count
Output (unused)	decoder_hidden, pre_acoustic_embeds — SeACo hotword pathway, not read by onnx-asr

The 560 dim input is the FunASR frontend: an 80 dim kaldi fbank of a waveform scaled to the int16 range, then a low frame rate stack of 7 frames with a hop of 6, then the am.mvn mean variance statistics. onnx-asr computes the fbank with its wespeaker preprocessor and applies the LFR stack and the CMVN in the runtime. Decoding is one argmax per logits row, stopping at </s> and never reading past token_num.

Streaming

A streaming Paraformer also exists upstream, using a different graph with encoder and decoder states, and needs a streaming runtime that onnx-asr does not have yet (upstream issue #21). Only the offline model is mirrored here.

Files

model.onnx (871 MB) and model_int8.onnx (245 MB). Both come straight from the sherpa-onnx repository; nothing was re-quantized here.

Parity with native FunASR

2 Mandarin, 2 English and 2 Cantonese clips (FLEURS cmn_hans_cn, en_us, yue_hant_hk), native FunASR 1.4.0 on the exact ModelScope checkpoint (dengcunqin/speech_seaco_paraformer_large_asr_nat-zh-cantonese-en-16k-common-vocab11666-pytorch) with frontend.dither = 0.

Clip	fp32	int8
zh_1	identical	identical
zh_2	identical	identical
en_1	identical	identical
en_2	identical	a few words differ (deliberately out-of-domain foreign-cuisine clip)
yue_1	identical	identical
yue_2	identical	identical

6 of 6 fp32 comparisons and 5 of 6 int8 comparisons are character identical to native FunASR. The one int8 difference is on the same deliberately hard, out-of-domain English clip that shows quantization noise in the other Paraformer mirrors in this collection. Cantonese, the language this checkpoint adds to the collection, is perfect in both precisions.

Speed

AMD Ryzen 5 7600, 6 cores, OMP_NUM_THREADS=6, CPU provider, batch 1, warm cache, best of 3 passes over the 6 test clips.

Precision	RTF
fp32	0.0196
int8	0.0148

License

Apache License 2.0. The ModelScope model card states License: Apache License 2.0. The sherpa-onnx mirror carries no license file of its own, which is why the ModelScope source was read directly rather than the mirror.

• Model: Alibaba DAMO Academy / FunASR, dengcunqin/speech_seaco_paraformer_large_asr_nat-zh-cantonese-en-16k-common-vocab11666-pytorch, Apache-2.0.
• ONNX export: sherpa-onnx by Fangjun Kuang (csukuangfj), Xiaomi.
• This repository only repackages those files for onnx-asr.

Deviations

• The LFR stack and the CMVN are in the runtime class, not in the graph, matching every other Paraformer mirror in this collection.
• No timestamps. Paraformer has no per-token frame index.
• No inverse text normalization and no homophone replacement.
• The streaming Paraformer is out of scope (see above).
• This checkpoint is SeACo-Paraformer (hotword biasing), used here without hotwords. The bias-pathway outputs are present in the graph but unread.