harshav commited on
Commit
4134f13
·
verified ·
1 Parent(s): ed684a5

Rewrite model card: YAML metadata, artifact-form definitions, file layout, attribution

Browse files
Files changed (1) hide show
  1. README.md +73 -22
README.md CHANGED
@@ -1,17 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # ARK-ASR-3B Core AI
2
 
3
  Apple Core AI conversions of [Audio8/ARK-ASR-3B](https://huggingface.co/Audio8/ARK-ASR-3B), made from source revision `1e28271b79edc97635783bea65abc89195a09ed3`.
4
 
5
- ARK-ASR combines a Whisper-style audio encoder, an MLP adapter, and a 36-layer Qwen2 decoder. This repo ships two artifact forms:
6
 
7
- - **`.aimodel`** — portable Core AI assets. The encoder runs anywhere Core AI does (macOS 27+); the decoder is an int8 weight-only (per-block-32) unified bundle with static `prefill` + `decode` entrypoints sharing one weight copy and a host-side KV cache.
8
- - **`.aimodelc`** — AOT-compiled variants for a specific Apple silicon architecture (`h16c` = M4 Max family; see `xcrun coreai-build list-architectures`). They skip runtime specialization entirely: measured 74 s → ~0 s model preparation versus specializing the `.aimodel` on first launch. The runtime **hard-rejects** mismatched architectures (`incompatibleCompiledAssetArchitecture`), so only install the variant matching your device. Other architectures can be produced from the `.aimodel` with `xcrun coreai-build compile --architecture <arch>`.
9
 
10
- ## Runtime
 
 
 
 
 
 
 
 
11
 
12
- These assets are **not** compatible with Apple's `CoreAISpeech` (`SpeechBundle` expects split encoder/decoder Whisper layouts) and are not GGUF — for `llama.cpp`-style runtimes see [harshav/ARK-ASR-3B-GGUF](https://huggingface.co/harshav/ARK-ASR-3B-GGUF) instead.
13
 
14
- `coreai-arkasr-conversion.tar.gz` is the complete conversion + verification toolkit used to produce and gate these artifacts (SHA-256: `8a24d7069ec64462703986445af5841541e626235cf03c53ece8d3aefc08d77f`), including the port's `STATE.md` with full reproduction steps:
15
 
16
  ```bash
17
  tar -xzf coreai-arkasr-conversion.tar.gz && cd ark_asr
@@ -21,23 +62,33 @@ uv run --python 3.12 --with-editable <coreai-models>/python export_unified.py --
21
  uv run --python 3.12 gate_static.py --unified --mode int8 --cache-len 1024
22
  ```
23
 
24
- The reference client is [VoiceInk](https://github.com/Beingpax/VoiceInk) (`Transcription/CoreAI/`): mel frontend, prompt splicing (`vocabSize + slot` audio injection), host KV cache, greedy decode. `runtime_config.json`, `mel_filters.f32`, and the tokenizer files are the exact inputs that client consumes.
25
 
26
  ## Files
27
 
28
- | File | Size | SHA-256 | Notes |
29
- | --- | ---: | --- | --- |
30
- | `encoder.aimodel/main.mlirb` | 1,329,458,070 | `14bfbb6d1503c6cf26686a3aa855614cd33a1d0cc686698788697f542fb0055c` | fp16 Whisper-tower + MLP adapter, mel `[1,128,3000]` → `audio_embeds [375,2048]` |
31
- | `decoder.aimodel/main.mlirb` | 3,572,562,869 | `e324f285d79421f18baea5213a3b021b813df7d15e009f01260b64517bd90b4d` | int8 static unified prefill+decode, KV state externalized |
32
- | `encoder.h16c.aimodelc/.../resources.bin` | 1,328,929,212 | `7ba10b5317d5fb1eee88df8ec3a318ac43984cfa25221bbd8f2f8dbde7172f1b` | AOT-compiled encoder, h16c only |
33
- | `decoder.h16c.aimodelc/.../resources.bin` | 12,966,484,252 | `88385079c215b51ad522b62c436a3a1ab72987f10d2d99128e0cb54d53ad871e` | AOT-compiled decoder, h16c only |
34
- | `coreai-arkasr-conversion.tar.gz` | 23,471 | `8a24d7069ec64462703986445af5841541e626235cf03c53ece8d3aefc08d77f` | Conversion + gate toolkit |
35
- | `runtime_config.json` | 355 | `2f7dd4e033cbc361fc46f735b68bcb0a23782e9ccd01e88c8c5422d660f9e4f1` | Prompt/config constants for clients |
36
- | `mel_filters.f32` | 102,912 | `c342ed6d06c90b42990bf9e4a3a07ca54f3f1422f7bc3c66b3177a774395df5a` | Exact slaney mel filterbank from ARK's feature extractor — do not recompute (HTK formula diverges ~24x) |
 
 
 
 
 
 
 
 
 
 
37
 
38
  ## Local validation
39
 
40
- Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `coreai-torch` 0.4.1, against an fp32 PyTorch golden of the official model. Pass condition is exact token-for-token equality, not similarity.
41
 
42
  | Stage | Result |
43
  | --- | --- |
@@ -46,19 +97,19 @@ Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `
46
 
47
  Performance on the same machine (12.75 s clip):
48
 
49
- | Metric | fp16 dynamic | **int8 static (this repo)** |
50
  | --- | ---: | ---: |
51
  | gate load | 62.7 s | **3.4 s** |
52
  | prefill | 1.75 s | **0.31 s** |
53
  | decode | 71.0 ms/tok | **51.8 ms/tok** |
54
  | app prewarm, fresh process | 127 s | **74 s** |
55
 
56
- Architecture note: ARK decodes autoregressively through a 36-layer 3B LLM, so ~52 ms/token is a bandwidth floor every framework hits (GGUF/llama.cpp included). Purpose-built ASR models (e.g. Cohere Transcribe) are structurally faster for short dictation.
57
 
58
  ## Long audio
59
 
60
- The model is a fixed 30-second-window architecture: the official feature extractor pads/trims mel to 3000 frames, and a 33.5 s probe confirms the reference implementation truncates at the window edge. Longer audio is a host-side concern — window the input (the GGUF runtime in `harshav/ARK-ASR-3B-GGUF` overlaps 2 s and stitches) and concatenate transcripts.
61
 
62
- ## License
63
 
64
- Apache-2.0, following the source model.
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Audio8/ARK-ASR-3B
4
+ pipeline_tag: automatic-speech-recognition
5
+ language:
6
+ - en
7
+ - zh
8
+ - de
9
+ - ja
10
+ - fr
11
+ - ko
12
+ - es
13
+ - pl
14
+ - it
15
+ - ro
16
+ - hu
17
+ - cs
18
+ - nl
19
+ - fi
20
+ - hr
21
+ - sk
22
+ - sl
23
+ - et
24
+ - lt
25
+ tags:
26
+ - coreai
27
+ - aimodel
28
+ - aimodelc
29
+ - arkasr
30
+ - speech
31
+ - audio
32
+ - apple-silicon
33
+ ---
34
+
35
  # ARK-ASR-3B Core AI
36
 
37
  Apple Core AI conversions of [Audio8/ARK-ASR-3B](https://huggingface.co/Audio8/ARK-ASR-3B), made from source revision `1e28271b79edc97635783bea65abc89195a09ed3`.
38
 
39
+ ARK-ASR combines a Whisper-style audio encoder, an MLP adapter, and a Qwen2.5 decoder. These files use Apple's Core AI runtime (macOS 27+); they are **not** GGUF and are **not** compatible with `llama.cpp`, `whisper.cpp`, or Apple's `CoreAISpeech` bundle layout. For GGUF runtimes see [harshav/ARK-ASR-3B-GGUF](https://huggingface.co/harshav/ARK-ASR-3B-GGUF).
40
 
41
+ ## Artifact forms
 
42
 
43
+ | Term | Meaning |
44
+ | --- | --- |
45
+ | `.aimodel` | Portable Core AI source asset. Runs on any macOS 27+ device; the runtime specializes it for your chip on first use (one-time, cached). |
46
+ | `.aimodelc` | AOT-compiled variant for one specific silicon architecture, produced by `xcrun coreai-build compile --architecture <arch>`. Skips runtime specialization entirely (measured 74 s → ~0 s of model preparation per app launch). The runtime **hard-rejects** mismatched chips with `incompatibleCompiledAssetArchitecture`, so only use the variant matching your device (`h16c` = M4 Max family; list yours with `xcrun coreai-build list-architectures`). Other architectures compile from the `.aimodel` in minutes. |
47
+ | `fp16` | Full-precision weights (encoder). |
48
+ | `int8` | Weight-only quantization, per-block-32 symmetric with clipping (decoder). SDPA, RoPE, RMSNorm, embeddings, and the LM head stay in fp16. |
49
+ | `static` | The prefill graph's prompt shape is baked (2 + 375 audio slots + 3 = 380 tokens, the constant 30 s window). A dynamic-shape prefill costs ~3 GB more compiled package and ~4x slower gate load for zero behavior change. |
50
+
51
+ The decoder is one unified bundle: `prefill` + `decode` entrypoints share a single weight copy, with the KV cache externalized and threaded host-side.
52
 
53
+ ## Runtime
54
 
55
+ `coreai-arkasr-conversion.tar.gz` is the complete conversion and verification toolkit used to produce and gate these artifacts (SHA-256: `8a24d7069ec64462703986445af5841541e626235cf03c53ece8d3aefc08d77f`), including the port's `STATE.md` with full reproduction steps:
56
 
57
  ```bash
58
  tar -xzf coreai-arkasr-conversion.tar.gz && cd ark_asr
 
62
  uv run --python 3.12 gate_static.py --unified --mode int8 --cache-len 1024
63
  ```
64
 
65
+ The reference client is [VoiceInk](https://github.com/Beingpax/VoiceInk) (`Transcription/CoreAI/`): mel frontend, prompt splicing (`vocabSize + slot` audio injection), host KV cache, greedy decode. `runtime_config.json`, `mel_filters.f32`, and the tokenizer files are the exact inputs that client consumes. Audio input is 16 kHz mono.
66
 
67
  ## Files
68
 
69
+ ```text
70
+ encoder.aimodel/ fp16 Whisper tower + MLP adapter
71
+ encoder.h16c.aimodelc/ AOT-compiled encoder (h16c / M4 Max only)
72
+ decoder.aimodel/ int8 static unified prefill+decode
73
+ decoder.h16c.aimodelc/ AOT-compiled decoder (h16c / M4 Max only)
74
+ runtime_config.json prompt/config constants for clients
75
+ mel_filters.f32 exact slaney mel filterbank from ARK's feature extractor
76
+ tokenizer*.json, vocab.json, merges.txt, added_tokens.json, special_tokens_map.json
77
+ coreai-arkasr-conversion.tar.gz conversion + gate toolkit
78
+ ```
79
+
80
+ | Payload | Size | SHA-256 |
81
+ | --- | ---: | --- |
82
+ | `encoder.aimodel/main.mlirb` | 1,329,458,070 bytes | `14bfbb6d1503c6cf26686a3aa855614cd33a1d0cc686698788697f542fb0055c` |
83
+ | `decoder.aimodel/main.mlirb` | 3,572,562,869 bytes | `e324f285d79421f18baea5213a3b021b813df7d15e009f01260b64517bd90b4d` |
84
+ | `encoder.h16c.aimodelc` resources.bin | 1,328,929,212 bytes | `7ba10b5317d5fb1eee88df8ec3a318ac43984cfa25221bbd8f2f8dbde7172f1b` |
85
+ | `decoder.h16c.aimodelc` resources.bin | 12,966,484,252 bytes | `88385079c215b51ad522b62c436a3a1ab72987f10d2d99128e0cb54d53ad871e` |
86
+
87
+ Do not recompute the mel filterbank: Whisper uses slaney-scaled normalized triangles (peak ≈ 0.042), and the textbook HTK formula diverges by ~24x, which measurably corrupts transcription. Use `mel_filters.f32` as shipped.
88
 
89
  ## Local validation
90
 
91
+ Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), `coreai-core` 1.0.0b2, `coreai-torch` 0.4.1, against an fp32 PyTorch golden of the official model. The pass condition is exact token-for-token equality, not similarity.
92
 
93
  | Stage | Result |
94
  | --- | --- |
 
97
 
98
  Performance on the same machine (12.75 s clip):
99
 
100
+ | Metric | fp16 dynamic | int8 static (this repo) |
101
  | --- | ---: | ---: |
102
  | gate load | 62.7 s | **3.4 s** |
103
  | prefill | 1.75 s | **0.31 s** |
104
  | decode | 71.0 ms/tok | **51.8 ms/tok** |
105
  | app prewarm, fresh process | 127 s | **74 s** |
106
 
107
+ Architecture note: ARK decodes autoregressively through a 36-layer 3B LLM, so ~52 ms/token is a bandwidth floor every framework hits, GGUF/llama.cpp included. Purpose-built ASR models (e.g. Cohere Transcribe) are structurally faster for short dictation.
108
 
109
  ## Long audio
110
 
111
+ The model is a fixed 30-second-window architecture: the official feature extractor pads or trims mel to 3000 frames, and a 33.5 s probe confirms the reference implementation truncates at the window edge. Longer audio is a host-side concern — window the input (the GGUF runtime in [harshav/ARK-ASR-3B-GGUF](https://huggingface.co/harshav/ARK-ASR-3B-GGUF) overlaps windows by 2 s and stitches) and concatenate transcripts.
112
 
113
+ ## License and attribution
114
 
115
+ The original model is Apache-2.0 licensed. See the [official model card](https://huggingface.co/Audio8/ARK-ASR-3B) and [AutoArk repository](https://github.com/AutoArk/open-audio-opd) for architecture, training, and upstream attribution.