ONNX export of Audio8/ARK-ASR-0.6B for onnx-asr

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+decoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+embed_tokens.onnx.data filter=lfs diff=lfs merge=lfs -text
+encoder.onnx.data filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,100 @@
+---
+license: apache-2.0
+base_model: Audio8/ARK-ASR-0.6B
+base_model_relation: quantized
+library_name: onnx-asr
+pipeline_tag: automatic-speech-recognition
+tags:
+  - onnx
+  - onnxruntime
+  - automatic-speech-recognition
+  - speech
+  - audio
+  - asr
+  - int8
+  - speech-llm
+language:
+  - zh
+  - en
+  - de
+  - ja
+  - fr
+  - ko
+  - es
+  - pl
+  - it
+  - ro
+  - hu
+  - cs
+  - nl
+  - fi
+  - hr
+  - sk
+  - sl
+  - et
+  - lt
+---
+
+# ARK-ASR-0.6B ONNX
+
+ONNX export of [Audio8/ARK-ASR-0.6B](https://huggingface.co/Audio8/ARK-ASR-0.6B) for
+[onnx-asr](https://github.com/istupakov/onnx-asr). All credit for the model goes to
+Audio8 (AutoArk AI). This repository only contains the converted graphs; the
+weights are the original ones.
+
+The model is a speech-LLM: a Whisper-large-v3-style audio encoder with rotary
+position embeddings, an MLP adapter that merges four encoder frames into one
+embedding, and a Qwen2 0.6B causal language model that writes the transcription.
+
+## Usage
+
+```sh
+pip install onnx-asr[cpu,hub]
+```
+
+```py
+import onnx_asr
+
+model = onnx_asr.load_model("speech-llm", "OpenVoiceOS/ARK-ASR-0.6B-onnx")
+print(model.recognize("audio.wav"))
+
+# int8 weights
+model = onnx_asr.load_model("speech-llm", "OpenVoiceOS/ARK-ASR-0.6B-onnx", quantization="int8")
+```
+
+## Files
+
+| File | Contents |
+| --- | --- |
+| `encoder.onnx` | audio encoder and MLP adapter, log-mel features in, LM embeddings out |
+| `embed_tokens.onnx` | token embedding table |
+| `decoder.onnx` | Qwen2 decoder with KV cache, logits out |
+| `*.int8.onnx` | dynamically quantized int8 weights |
+| `config.json` | model type, prompt token ids, suppressed token ids |
+| `vocab.json` | tokenizer vocabulary for detokenization |
+
+## Graph contract
+
+| Graph | Inputs | Outputs |
+| --- | --- | --- |
+| `encoder.onnx` | `input_features (1, 128, frames)` | `audio_embeds (1, frames/8, 896)` |
+| `embed_tokens.onnx` | `input_ids (1, S)` | `inputs_embeds (1, S, 896)` |
+| `decoder.onnx` | `inputs_embeds (1, S, 896)`, `attn_bias (1, 1, S, P+S)`, `position_ids (1, S)`, `past_key_values.{0..23}.{key,value} (1, 2, P, 64)` | `logits (1, S, 163958)`, `present.{0..23}.{key,value} (1, 2, P+S, 64)` |
+
+## Accuracy
+
+Four FLEURS clips (2 English, 2 Mandarin), greedy decoding, compared against the
+PyTorch model in float32:
+
+* fp32 ONNX: 4 of 4 transcriptions identical to PyTorch, character for character.
+* int8 ONNX: 3 of 4 identical. One Mandarin clip stops early and loses the last
+  clause.
+
+Speed on a 12-core CPU under load: RTF 0.35 to 0.61 (fp32) and 0.09 to 0.29
+(int8).
+
+## Licence
+
+Apache 2.0, the same licence as the source model. The model was published by
+Audio8; see the [source repository](https://huggingface.co/Audio8/ARK-ASR-0.6B)
+and the paper [arXiv:2605.28139](https://arxiv.org/abs/2605.28139).

decoder.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2541ddceec2eb5c1d140d3366c1273856a444d15123ebdd5a4949ca66228ef3a
+size 506552677

decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a150f6710919e11c1ba51ea8e5fc946acbc8489212334c9c159b36e288e230f7
+size 995553

decoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fd5435175ce81cdce6f4543b09f194e2ac97e5e0bb6089660234e2b567e3676
+size 2019193344

embed_tokens.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b3d5c49e7aa511b861b48a37c3dd8f65b688d11e2dc77d97dc073d66eb41eeec
+size 146906888

embed_tokens.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26d72739d9946bb606fc36c0436d874a0fe6e3d96a31f855e91950fc651e8d6c
+size 277

embed_tokens.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2102ed49ea4f1181092468e3c24a253955692cd0bc51a1e4d26d83ed70114db
+size 587625472

encoder.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d449e6df0b2eb519f4c6a7e2151e4428f35b116c6edb8aefa8709fd60bc64e0
+size 649600736

encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03b9fc7f4a35ab742a611e91354967ce50ba8aa5d9ace981f550682eb7da1d46
+size 1496674

encoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:57501ca6aedc745b200a2fcca3e82b77b4d06f0b2b8c0a72e8e007cff5a76136
+size 2583329280