Rewrite model card: YAML metadata, artifact-form definitions, file layout, attribution
Browse files
README.md
CHANGED
|
@@ -1,17 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# ARK-ASR-3B Core AI
|
| 2 |
|
| 3 |
Apple Core AI conversions of [Audio8/ARK-ASR-3B](https://huggingface.co/Audio8/ARK-ASR-3B), made from source revision `1e28271b79edc97635783bea65abc89195a09ed3`.
|
| 4 |
|
| 5 |
-
ARK-ASR combines a Whisper-style audio encoder, an MLP adapter, and a
|
| 6 |
|
| 7 |
-
|
| 8 |
-
- **`.aimodelc`** — AOT-compiled variants for a specific Apple silicon architecture (`h16c` = M4 Max family; see `xcrun coreai-build list-architectures`). They skip runtime specialization entirely: measured 74 s → ~0 s model preparation versus specializing the `.aimodel` on first launch. The runtime **hard-rejects** mismatched architectures (`incompatibleCompiledAssetArchitecture`), so only install the variant matching your device. Other architectures can be produced from the `.aimodel` with `xcrun coreai-build compile --architecture <arch>`.
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
-
|
| 13 |
|
| 14 |
-
`coreai-arkasr-conversion.tar.gz` is the complete conversion
|
| 15 |
|
| 16 |
```bash
|
| 17 |
tar -xzf coreai-arkasr-conversion.tar.gz && cd ark_asr
|
|
@@ -21,23 +62,33 @@ uv run --python 3.12 --with-editable <coreai-models>/python export_unified.py --
|
|
| 21 |
uv run --python 3.12 gate_static.py --unified --mode int8 --cache-len 1024
|
| 22 |
```
|
| 23 |
|
| 24 |
-
The reference client is [VoiceInk](https://github.com/Beingpax/VoiceInk) (`Transcription/CoreAI/`): mel frontend, prompt splicing (`vocabSize + slot` audio injection), host KV cache, greedy decode. `runtime_config.json`, `mel_filters.f32`, and the tokenizer files are the exact inputs that client consumes.
|
| 25 |
|
| 26 |
## Files
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
## Local validation
|
| 39 |
|
| 40 |
-
Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `coreai-torch` 0.4.1, against an fp32 PyTorch golden of the official model.
|
| 41 |
|
| 42 |
| Stage | Result |
|
| 43 |
| --- | --- |
|
|
@@ -46,19 +97,19 @@ Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `
|
|
| 46 |
|
| 47 |
Performance on the same machine (12.75 s clip):
|
| 48 |
|
| 49 |
-
| Metric | fp16 dynamic |
|
| 50 |
| --- | ---: | ---: |
|
| 51 |
| gate load | 62.7 s | **3.4 s** |
|
| 52 |
| prefill | 1.75 s | **0.31 s** |
|
| 53 |
| decode | 71.0 ms/tok | **51.8 ms/tok** |
|
| 54 |
| app prewarm, fresh process | 127 s | **74 s** |
|
| 55 |
|
| 56 |
-
Architecture note: ARK decodes autoregressively through a 36-layer 3B LLM, so ~52 ms/token is a bandwidth floor every framework hits
|
| 57 |
|
| 58 |
## Long audio
|
| 59 |
|
| 60 |
-
The model is a fixed 30-second-window architecture: the official feature extractor pads
|
| 61 |
|
| 62 |
-
## License
|
| 63 |
|
| 64 |
-
Apache-2.0
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Audio8/ARK-ASR-3B
|
| 4 |
+
pipeline_tag: automatic-speech-recognition
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
- zh
|
| 8 |
+
- de
|
| 9 |
+
- ja
|
| 10 |
+
- fr
|
| 11 |
+
- ko
|
| 12 |
+
- es
|
| 13 |
+
- pl
|
| 14 |
+
- it
|
| 15 |
+
- ro
|
| 16 |
+
- hu
|
| 17 |
+
- cs
|
| 18 |
+
- nl
|
| 19 |
+
- fi
|
| 20 |
+
- hr
|
| 21 |
+
- sk
|
| 22 |
+
- sl
|
| 23 |
+
- et
|
| 24 |
+
- lt
|
| 25 |
+
tags:
|
| 26 |
+
- coreai
|
| 27 |
+
- aimodel
|
| 28 |
+
- aimodelc
|
| 29 |
+
- arkasr
|
| 30 |
+
- speech
|
| 31 |
+
- audio
|
| 32 |
+
- apple-silicon
|
| 33 |
+
---
|
| 34 |
+
|
| 35 |
# ARK-ASR-3B Core AI
|
| 36 |
|
| 37 |
Apple Core AI conversions of [Audio8/ARK-ASR-3B](https://huggingface.co/Audio8/ARK-ASR-3B), made from source revision `1e28271b79edc97635783bea65abc89195a09ed3`.
|
| 38 |
|
| 39 |
+
ARK-ASR combines a Whisper-style audio encoder, an MLP adapter, and a Qwen2.5 decoder. These files use Apple's Core AI runtime (macOS 27+); they are **not** GGUF and are **not** compatible with `llama.cpp`, `whisper.cpp`, or Apple's `CoreAISpeech` bundle layout. For GGUF runtimes see [harshav/ARK-ASR-3B-GGUF](https://huggingface.co/harshav/ARK-ASR-3B-GGUF).
|
| 40 |
|
| 41 |
+
## Artifact forms
|
|
|
|
| 42 |
|
| 43 |
+
| Term | Meaning |
|
| 44 |
+
| --- | --- |
|
| 45 |
+
| `.aimodel` | Portable Core AI source asset. Runs on any macOS 27+ device; the runtime specializes it for your chip on first use (one-time, cached). |
|
| 46 |
+
| `.aimodelc` | AOT-compiled variant for one specific silicon architecture, produced by `xcrun coreai-build compile --architecture <arch>`. Skips runtime specialization entirely (measured 74 s → ~0 s of model preparation per app launch). The runtime **hard-rejects** mismatched chips with `incompatibleCompiledAssetArchitecture`, so only use the variant matching your device (`h16c` = M4 Max family; list yours with `xcrun coreai-build list-architectures`). Other architectures compile from the `.aimodel` in minutes. |
|
| 47 |
+
| `fp16` | Full-precision weights (encoder). |
|
| 48 |
+
| `int8` | Weight-only quantization, per-block-32 symmetric with clipping (decoder). SDPA, RoPE, RMSNorm, embeddings, and the LM head stay in fp16. |
|
| 49 |
+
| `static` | The prefill graph's prompt shape is baked (2 + 375 audio slots + 3 = 380 tokens, the constant 30 s window). A dynamic-shape prefill costs ~3 GB more compiled package and ~4x slower gate load for zero behavior change. |
|
| 50 |
+
|
| 51 |
+
The decoder is one unified bundle: `prefill` + `decode` entrypoints share a single weight copy, with the KV cache externalized and threaded host-side.
|
| 52 |
|
| 53 |
+
## Runtime
|
| 54 |
|
| 55 |
+
`coreai-arkasr-conversion.tar.gz` is the complete conversion and verification toolkit used to produce and gate these artifacts (SHA-256: `8a24d7069ec64462703986445af5841541e626235cf03c53ece8d3aefc08d77f`), including the port's `STATE.md` with full reproduction steps:
|
| 56 |
|
| 57 |
```bash
|
| 58 |
tar -xzf coreai-arkasr-conversion.tar.gz && cd ark_asr
|
|
|
|
| 62 |
uv run --python 3.12 gate_static.py --unified --mode int8 --cache-len 1024
|
| 63 |
```
|
| 64 |
|
| 65 |
+
The reference client is [VoiceInk](https://github.com/Beingpax/VoiceInk) (`Transcription/CoreAI/`): mel frontend, prompt splicing (`vocabSize + slot` audio injection), host KV cache, greedy decode. `runtime_config.json`, `mel_filters.f32`, and the tokenizer files are the exact inputs that client consumes. Audio input is 16 kHz mono.
|
| 66 |
|
| 67 |
## Files
|
| 68 |
|
| 69 |
+
```text
|
| 70 |
+
encoder.aimodel/ fp16 Whisper tower + MLP adapter
|
| 71 |
+
encoder.h16c.aimodelc/ AOT-compiled encoder (h16c / M4 Max only)
|
| 72 |
+
decoder.aimodel/ int8 static unified prefill+decode
|
| 73 |
+
decoder.h16c.aimodelc/ AOT-compiled decoder (h16c / M4 Max only)
|
| 74 |
+
runtime_config.json prompt/config constants for clients
|
| 75 |
+
mel_filters.f32 exact slaney mel filterbank from ARK's feature extractor
|
| 76 |
+
tokenizer*.json, vocab.json, merges.txt, added_tokens.json, special_tokens_map.json
|
| 77 |
+
coreai-arkasr-conversion.tar.gz conversion + gate toolkit
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
| Payload | Size | SHA-256 |
|
| 81 |
+
| --- | ---: | --- |
|
| 82 |
+
| `encoder.aimodel/main.mlirb` | 1,329,458,070 bytes | `14bfbb6d1503c6cf26686a3aa855614cd33a1d0cc686698788697f542fb0055c` |
|
| 83 |
+
| `decoder.aimodel/main.mlirb` | 3,572,562,869 bytes | `e324f285d79421f18baea5213a3b021b813df7d15e009f01260b64517bd90b4d` |
|
| 84 |
+
| `encoder.h16c.aimodelc` resources.bin | 1,328,929,212 bytes | `7ba10b5317d5fb1eee88df8ec3a318ac43984cfa25221bbd8f2f8dbde7172f1b` |
|
| 85 |
+
| `decoder.h16c.aimodelc` resources.bin | 12,966,484,252 bytes | `88385079c215b51ad522b62c436a3a1ab72987f10d2d99128e0cb54d53ad871e` |
|
| 86 |
+
|
| 87 |
+
Do not recompute the mel filterbank: Whisper uses slaney-scaled normalized triangles (peak ≈ 0.042), and the textbook HTK formula diverges by ~24x, which measurably corrupts transcription. Use `mel_filters.f32` as shipped.
|
| 88 |
|
| 89 |
## Local validation
|
| 90 |
|
| 91 |
+
Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `coreai-torch` 0.4.1, against an fp32 PyTorch golden of the official model. The pass condition is exact token-for-token equality, not similarity.
|
| 92 |
|
| 93 |
| Stage | Result |
|
| 94 |
| --- | --- |
|
|
|
|
| 97 |
|
| 98 |
Performance on the same machine (12.75 s clip):
|
| 99 |
|
| 100 |
+
| Metric | fp16 dynamic | int8 static (this repo) |
|
| 101 |
| --- | ---: | ---: |
|
| 102 |
| gate load | 62.7 s | **3.4 s** |
|
| 103 |
| prefill | 1.75 s | **0.31 s** |
|
| 104 |
| decode | 71.0 ms/tok | **51.8 ms/tok** |
|
| 105 |
| app prewarm, fresh process | 127 s | **74 s** |
|
| 106 |
|
| 107 |
+
Architecture note: ARK decodes autoregressively through a 36-layer 3B LLM, so ~52 ms/token is a bandwidth floor every framework hits, GGUF/llama.cpp included. Purpose-built ASR models (e.g. Cohere Transcribe) are structurally faster for short dictation.
|
| 108 |
|
| 109 |
## Long audio
|
| 110 |
|
| 111 |
+
The model is a fixed 30-second-window architecture: the official feature extractor pads or trims mel to 3000 frames, and a 33.5 s probe confirms the reference implementation truncates at the window edge. Longer audio is a host-side concern — window the input (the GGUF runtime in [harshav/ARK-ASR-3B-GGUF](https://huggingface.co/harshav/ARK-ASR-3B-GGUF) overlaps windows by 2 s and stitches) and concatenate transcripts.
|
| 112 |
|
| 113 |
+
## License and attribution
|
| 114 |
|
| 115 |
+
The original model is Apache-2.0 licensed. See the [official model card](https://huggingface.co/Audio8/ARK-ASR-3B) and [AutoArk repository](https://github.com/AutoArk/open-audio-opd) for architecture, training, and upstream attribution.
|