Can i export onnx models and use it in sherpa-onnx
Hi, thanks for making this model public.
Since access to it is currently restricted, would it be okay if I export it to sherpa-onnx and share the ONNX version of this repository for others to use?
Yeah, I also wonder... Since access is currently restricted, I was waiting for this to be fully public... 🙃
Based on my tests, it works really nice by the way and it's really fast even on CPU! Ty to make this model as public.
@csukuangfj , @altunenes - The model is still WIP, mainly WER improvements for some of the languages. It will be released very soon with slightly better quality.
@Amargolin curious about languages support, is there a list or something or not yet ?
its interesting because model card officially certifies 36 languages. But if I inspect the model's internal prompt_dictionary actually has slots for more I mean like 100+ (including e.g. Quechua, Maori, Hawaiian langs too)...
it's also worth noting that the model seems quite sensitive to accents. I tested a real world Swedish recordings (couple of, not that much) last night with two speakers both speaking Swedish, but one is a native German speaker with a clear German accent (naturally 😁 ). With auto detection, the model occasionally transcribed his Swedish as German instead. Not sure if that's a feature (the LID network honestly catching the non-native pronunciation) or a limitation... If I gave the language hint beforehand, of course there would be no problem. It might be a very rare occurrence of course I haven't tested that much with that...
@MrEzzat -
great -> reasonable WER
English US
English en-GB
Spanish US
Spanish es-ES
French
French fr-CA
Italian
Portuguese BR
Portuguese PT
Russian
Dutch
German
Polish
Czech
Arabic
Hindi
Japanese
Korean
Vietnamese
Turkish
Norwegian Bokmal
Hebrew
Danish
Swedish
Bulgarian
Finnish
Croatian
Slovak
Ukrainian
Additional langs added, but can be used as tokenizers supported:
Thai
Norwegian Nynorsk
Chinese Mandarin
Greek
Hungarian
Romanian
Estonian
Lithuanian
Latvian
Maltese
Slovenian
next stpes will be to add fine tuning recipes/skills
Hi @Amargolin what is expected date of general availability?
Been waiting to test out model for Hindi language in our production setting.
@snehmehta in the next couple of days. suggest to test the "vanilla" model and then run a short fine tuning task (no more than a 50$ spend on an L40S) to experiment with accuracy lift per langs of interest.
@Amargolin curious about languages support, is there a list or something or not yet ?
its interesting because model card officially certifies 36 languages. But if I inspect the model's internal prompt_dictionary actually has slots for more I mean like 100+ (including e.g. Quechua, Maori, Hawaiian langs too)...
it's also worth noting that the model seems quite sensitive to accents. I tested a real world Swedish recordings (couple of, not that much) last night with two speakers both speaking Swedish, but one is a native German speaker with a clear German accent (naturally 😁 ). With auto detection, the model occasionally transcribed his Swedish as German instead. Not sure if that's a feature (the LID network honestly catching the non-native pronunciation) or a limitation... If I gave the language hint beforehand, of course there would be no problem. It might be a very rare occurrence of course I haven't tested that much with that...
Correction: I tested today with the new weights (on cpu, via onnx) handle code switched / accented audio much better. This time I tested in more deeply on my personal bench data... even when the German accented speaker is talking, the model still produces Swedish word for word correctly. There's still a small German bleed near the end when the German speaker is talking, but yeah, that's a normal behavior...
This model is really cool and solves many problems. Thank you!! 🙏
@altunenes - try fine tuning and adapting the languages balance :
Blog - https://huggingface.co/blog/nvidia/fine-tuning-nemotron-35-asr
Repo - https://github.com/nvidia-riva/tutorials/blob/main/asr-finetune-nemotron-3.5-asr-streaming-prompt.ipynb
@csukuangfj @altunenes : We'd love to use this when it's exported to sherpa-onnx. Is there an ETA?
hey Ialready exported it:
https://huggingface.co/altunenes/parakeet-rs/tree/main/nemotron-3.5-asr-streaming-0.6b-onnx
usage (rust demo):
https://github.com/altunenes/parakeet-rs/blob/master/examples/streaming.rs
you can also use my onnx export script here 🙂:
https://github.com/altunenes/parakeet-rs/blob/master/scripts/export_nemotron_streaming_multilingual.py
@csukuangfj @altunenes : We'd love to use this when it's exported to sherpa-onnx. Is there an ETA?
Please see https://github.com/k2-fsa/sherpa-onnx/issues/3664