Upload Zipformer CR-CTC transducer XL 290M checkpoint

.eval_results/open_asr_leaderboard.yaml

@@ -0,0 +1,72 @@
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: mean_wer
+  value: 5.64
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: ami_wer
+  value: 11.8
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: earnings22_wer
+  value: 7.73
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: gigaspeech_wer
+  value: 8.35
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: librispeech_clean_wer
+  value: 1.29
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: librispeech_other_wer
+  value: 3.03
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: spgispeech_wer
+  value: 1.65
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio
+- dataset:
+    id: hf-audio/open-asr-leaderboard
+    task_id: voxpopuli_wer
+  value: 5.64
+  date: '2026-07-09'
+  source:
+    url: https://huggingface.co/hf-audio
+    name: open-asr-leaderboard
+    user: hf-audio

README.md

@@ -0,0 +1,217 @@
+---
+language:
+- en
+license: apache-2.0
+library_name: k2
+pipeline_tag: automatic-speech-recognition
+datasets:
+- esb/datasets
+metrics:
+- wer
+tags:
+- automatic-speech-recognition
+- speech
+- audio
+- zipformer
+- transducer
+- cr-ctc
+- k2-fsa
+- offline-asr
+---
+
+# Zipformer-cr-ctc-transducer-XL-290M
+
+Offline English ASR model based on the Icefall/K2 pruned transducer Zipformer
+recipe. The model was trained with a transducer objective and CR-CTC
+(Consistency-Regularized CTC) auxiliary training, and is intended for Icefall
+transducer decoding.
+
+## Files
+
+- `model.pt`: Zipformer CR-CTC transducer checkpoint
+- `bpe.model`: SentencePiece BPE model
+- `tokens.txt`: Icefall token table exported from `bpe.model`
+- `config.yaml`: model architecture, feature extraction, tokenizer, Zipformer
+  model, decoding settings, and Hugging Face Hub download-metrics query file
+
+## Evaluation
+
+Open ASR Leaderboard English short-form result, decoded with
+`modified_beam_search` and beam size 6:
+
+| Metric | Value |
+| --- | ---: |
+| Average WER | 5.64 |
+| Parameters | 288M |
+
+Dataset WERs:
+
+| Dataset | WER |
+| --- | ---: |
+| AMI | 11.80 |
+| Earnings22 | 7.73 |
+| GigaSpeech | 8.35 |
+| LibriSpeech Clean | 1.29 |
+| LibriSpeech Other | 3.03 |
+| SPGISpeech | 1.65 |
+| VoxPopuli | 5.64 |
+| **Average** | **5.64** |
+
+## Training Data
+
+The model was trained on a combined English training mixture built from the
+training portions of the datasets below.
+
+| Dataset | Train Hours | Source |
+| --- | ---: | --- |
+| LibriSpeech | 960.0 | ESB datasets |
+| Earnings-22 | 105.0 | ESB datasets |
+| AMI Meeting Corpus | 78.0 | ESB datasets |
+| Common Voice Scripted Speech 25.0 - English | ~1,679.0 | Mozilla Data Collective |
+| Common Voice Spontaneous Speech 3.0 - English | ~3.6 | Mozilla Data Collective |
+| GigaSpeech XL | 10,000.0 | ESB datasets |
+| SPGISpeech | 4,900.0 | ESB datasets |
+| TED-LIUM Release 3 | 454.0 | ESB datasets |
+| VoxPopuli | 523.0 | ESB datasets |
+| Total | ~18,700 | ESB datasets and Mozilla Data Collective |
+
+Common Voice Scripted Speech 25.0 - English and Common Voice Spontaneous Speech
+3.0 - English are the English scripted and spontaneous Common Voice releases
+checked on May 12, 2026. Mozilla Data Collective lists them with March 30, 2026
+and March 22, 2026 release dates, respectively.
+
+For data normalization, the training transcripts were processed with direct LLM
+normalization using a self-hosted GLM-4.7 setup, plus custom agentic workflows
+built in collaboration with Claude Code 4.7 and Codex 5.4.
+
+## Training
+
+The model was trained with the k2-fsa/Icefall framework, PyTorch 2.10, and
+CUDA 13.0 on 8 NVIDIA B300 GPUs using bf16 automatic mixed precision. The final
+released checkpoint is an adaptive checkpoint soup selected from the offline
+extra-large Zipformer CR-CTC transducer run.
+
+## Usage With Icefall
+
+From an Icefall checkout, download this model repo locally:
+
+```bash
+huggingface-cli download soundsgoodai/Zipformer-cr-ctc-transducer-XL-290M \
+  --local-dir Zipformer-cr-ctc-transducer-XL-290M
+```
+
+Run offline decoding through Icefall. Input audio must already be 16 kHz:
+
+```bash
+cd icefall/egs/librispeech/ASR
+
+PYTHONPATH=../../.. python zipformer/pretrained.py \
+  --checkpoint /path/to/Zipformer-cr-ctc-transducer-XL-290M/model.pt \
+  --tokens /path/to/Zipformer-cr-ctc-transducer-XL-290M/tokens.txt \
+  --method modified_beam_search \
+  --beam-size 6 \
+  --num-encoder-layers "2,2,4,5,4,2" \
+  --downsampling-factor "1,2,4,8,4,2" \
+  --feedforward-dim "512,1024,2048,3072,2048,1024" \
+  --num-heads "4,4,4,8,4,4" \
+  --encoder-dim "192,384,768,1024,768,384" \
+  --encoder-unmasked-dim "192,192,320,384,320,192" \
+  --query-head-dim "32" \
+  --value-head-dim "12" \
+  --pos-head-dim "4" \
+  --pos-dim 48 \
+  --cnn-module-kernel "31,31,15,15,15,31" \
+  --decoder-dim 512 \
+  --joiner-dim 512 \
+  --context-size 2 \
+  --causal false \
+  --chunk-size "16,32,64,-1" \
+  --left-context-frames "64,128,256,-1" \
+  --use-transducer true \
+  --use-ctc false \
+  --use-attention-decoder false \
+  --use-cr-ctc true \
+  /path/to/audio_1.wav \
+  /path/to/audio_2.wav
+```
+
+The architecture and decoding values above are also recorded in `config.yaml`.
+
+## Decoding Methods
+
+The model supports the following Icefall decoding methods:
+
+- `greedy_search`
+- `modified_beam_search`
+- `fast_beam_search`
+
+The reported Open ASR Leaderboard result uses `modified_beam_search` with beam
+size 6.
+
+## Feature Extraction And Resampling
+
+For this model, use Kaldi-style fbank features. `kaldifeat` and
+`kaldi-native-fbank` are the recommended feature extraction backends.
+
+Audio should be mono 16 kHz before feature extraction. For sample-rate
+conversion, `audioop.ratecv` is recommended because it matches the resampling
+path used for evaluation. On Python 3.13 and newer, use an `audioop`-compatible
+package such as `audioop-lts`.
+
+## Output Formatting
+
+The model emits normalized English text with punctuation and capitalization. It
+does not automatically capitalize the first word of every sentence unless that
+word is normally capitalized, such as a proper noun, honorific, acronym, or
+similar named expression.
+
+The model also normalizes common written forms, including numbers, dates, and
+currency, to digit-based forms.
+
+## Open ASR Leaderboard Evaluation
+
+The `open_asr_leaderboard/soundsgoodai` runner downloads this model repo,
+resamples audio to 16 kHz with `audioop`, computes fbank features with
+`kaldi-native-fbank`, and decodes with Icefall `modified_beam_search`.
+
+```bash
+cd open_asr_leaderboard/soundsgoodai
+bash run_zipformer.sh
+```
+
+## Notes
+
+- The model expects 16 kHz mono audio.
+- The packaged decoding configuration uses `modified_beam_search` with beam
+  size 6.
+- The checkpoint is an offline ASR model and is not intended for streaming
+  decoding.
+- The model includes CR-CTC training heads, so Icefall loading should use
+  `--use-cr-ctc true`.
+
+## References
+
+- Zipformer paper: Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun
+  Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. "Zipformer: A
+  faster and better encoder for automatic speech recognition."
+  https://arxiv.org/abs/2310.11230
+- CR-CTC paper: Zengwei Yao, Wei Kang, Xiaoyu Yang, Fangjun Kuang, Liyong Guo,
+  Han Zhu, Zengrui Jin, Zhaoqing Li, Long Lin, and Daniel Povey.
+  "CR-CTC: Consistency regularization on CTC for improved speech recognition."
+  https://arxiv.org/abs/2410.05101
+- Icefall: https://github.com/k2-fsa/icefall
+- k2: https://github.com/k2-fsa/k2
+- kaldifeat: https://csukuangfj.github.io/kaldifeat/intro.html
+- kaldi-native-fbank: https://github.com/csukuangfj/kaldi-native-fbank
+- Python `audioop.ratecv`: https://docs.python.org/3.11/library/audioop.html#audioop.ratecv
+- audioop-lts for Python 3.13+: https://pypi.org/project/audioop-lts/
+- ESB datasets: https://huggingface.co/datasets/esb/datasets
+- LibriSpeech: https://www.openslr.org/12/
+- Earnings22: https://github.com/revdotcom/speech-datasets/tree/main/earnings22
+- AMI Meeting Corpus: https://www.idiap.ch/webarchives/sites/www.amiproject.org/ami-scientific-portal/meeting-corpus/
+- Common Voice Scripted Speech 25.0 - English: https://mozilladatacollective.com/datasets/cmndapwry02jnmh07dyo46mot
+- Common Voice Spontaneous Speech 3.0 - English: https://datacollective.mozillafoundation.org/datasets/cmn1pv5hi00uto1072y1074y7
+- GigaSpeech: https://github.com/SpeechColab/GigaSpeech
+- SPGISpeech: https://huggingface.co/datasets/kensho/spgispeech
+- TED-LIUM Release 3: https://lium.univ-lemans.fr/en/ted-lium3/
+- VoxPopuli: https://github.com/facebookresearch/voxpopuli

bpe.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:401a2a42bdac1b8e927316755e5cd5ef626acf3b05d17fe3dc4033f638f46654
+size 247388

config.yaml

@@ -0,0 +1,47 @@
+model_type: zipformer-transducer
+library_name: k2
+model_name: Zipformer-cr-ctc-transducer-XL-290M
+min_encoder_input_frames: 9
+file: model.pt
+tokenizer: bpe.model
+decoding:
+  method: modified_beam_search
+  beam_size: 6
+feature_opts:
+  frame_opts:
+    samp_freq: 16000
+    frame_shift_ms: 10
+    frame_length_ms: 25
+    dither: 0.0
+    preemph_coeff: 0.97
+    window_type: povey
+    blackman_coeff: 0.42
+    snip_edges: false
+  mel_opts:
+    num_bins: 80
+    low_freq: 20
+    high_freq: 7600
+model_params:
+  feature_dim: 80
+  subsampling_factor: 4
+  num_encoder_layers: 2,2,4,5,4,2
+  downsampling_factor: 1,2,4,8,4,2
+  feedforward_dim: 512,1024,2048,3072,2048,1024
+  num_heads: 4,4,4,8,4,4
+  encoder_dim: 192,384,768,1024,768,384
+  encoder_unmasked_dim: 192,192,320,384,320,192
+  query_head_dim: '32'
+  value_head_dim: '12'
+  pos_head_dim: '4'
+  pos_dim: 48
+  cnn_module_kernel: 31,31,15,15,15,31
+  decoder_dim: 512
+  joiner_dim: 512
+  context_size: 2
+  causal: false
+  chunk_size: 16,32,64,-1
+  left_context_frames: 64,128,256,-1
+  use_transducer: true
+  use_ctc: false
+  use_attention_decoder: false
+  use_cr_ctc: true

model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f78cbc426b67f2301044b0591685550aaaf748e9a7cbf343068450dba24e9425
+size 1155177039

tokens.txt

@@ -0,0 +1,512 @@
+<blk> 0
+<sos/eos> 1
+s 2
+. 3
+t 4
+▁ 5
+▁the 6
+, 7
+a 8
+▁a 9
+e 10
+▁to 11
+▁and 12
+d 13
+n 14
+ing 15
+o 16
+▁of 17
+u 18
+▁in 19
+y 20
+i 21
+ed 22
+' 23
+m 24
+c 25
+▁that 26
+p 27
+re 28
+▁s 29
+▁we 30
+r 31
+g 32
+▁I 33
+er 34
+al 35
+ar 36
+▁you 37
+▁it 38
+le 39
+▁is 40
+or 41
+b 42
+▁re 43
+ly 44
+f 45
+ri 46
+▁for 47
+th 48
+k 49
+▁be 50
+l 51
+w 52
+in 53
+▁so 54
+▁on 55
+▁f 56
+an 57
+te 58
+▁he 59
+es 60
+▁was 61
+▁b 62
+it 63
+▁with 64
+▁this 65
+ve 66
+on 67
+h 68
+▁c 69
+ur 70
+ce 71
+▁as 72
+▁p 73
+ra 74
+▁our 75
+ic 76
+ation 77
+▁w 78
+▁have 79
+▁are 80
+ent 81
+en 82
+ll 83
+▁de 84
+▁do 85
+se 86
+? 87
+▁but 88
+ct 89
+nd 90
+ch 91
+ment 92
+▁e 93
+ro 94
+il 95
+▁not 96
+li 97
+ter 98
+▁con 99
+ver 100
+▁me 101
+- 102
+la 103
+▁at 104
+▁what 105
+▁they 106
+vi 107
+▁or 108
+el 109
+ate 110
+▁S 111
+▁mo 112
+ad 113
+▁ma 114
+▁M 115
+▁ex 116
+▁can 117
+ion 118
+▁from 119
+▁there 120
+ne 121
+me 122
+▁ca 123
+▁all 124
+▁like 125
+▁his 126
+ck 127
+v 128
+ci 129
+▁us 130
+▁co 131
+▁go 132
+ge 133
+▁an 134
+▁fa 135
+▁by 136
+▁about 137
+lo 138
+▁C 139
+▁know 140
+ir 141
+▁some 142
+▁se 143
+ng 144
+▁had 145
+pe 146
+▁will 147
+▁lo 148
+un 149
+▁if 150
+▁up 151
+▁no 152
+▁think 153
+▁one 154
+▁my 155
+▁see 156
+▁A 157
+ol 158
+▁just 159
+▁out 160
+im 161
+ig 162
+et 163
+om 164
+ies 165
+▁which 166
+age 167
+▁pro 168
+▁more 169
+▁ho 170
+▁po 171
+▁ch 172
+▁pa 173
+ive 174
+x 175
+▁would 176
+▁B 177
+ity 178
+▁d 179
+▁year 180
+▁who 181
+▁your 182
+▁she 183
+▁were 184
+j 185
+▁P 186
+id 187
+▁when 188
+▁la 189
+▁mi 190
+▁man 191
+ist 192
+ers 193
+ul 194
+tion 195
+▁how 196
+▁her 197
+ther 198
+▁very 199
+end 200
+able 201
+▁now 202
+0 203
+▁pre 204
+de 205
+▁has 206
+▁time 207
+hi 208
+▁bo 209
+▁ha 210
+z 211
+▁their 212
+▁been 213
+ous 214
+▁com 215
+ight 216
+▁get 217
+▁D 218
+▁also 219
+▁people 220
+▁fi 221
+lu 222
+▁than 223
+est 224
+▁en 225
+mp 226
+▁li 227
+▁comp 228
+▁G 229
+▁sta 230
+ance 231
+▁other 232
+▁un 233
+ry 234
+ff 235
+▁T 236
+▁look 237
+ke 238
+▁these 239
+▁ne 240
+▁over 241
+ally 242
+ition 243
+▁work 244
+L 245
+▁going 246
+▁any 247
+▁don 248
+ld 249
+▁ra 250
+▁well 251
+▁ba 252
+▁because 253
+▁sp 254
+ction 255
+H 256
+ated 257
+ho 258
+qui 259
+J 260
+▁into 261
+▁ah 262
+▁dis 263
+▁E 264
+▁say 265
+▁really 266
+1 267
+▁where 268
+▁want 269
+▁F 270
+ical 271
+▁right 272
+low 273
+ful 274
+man 275
+▁op 276
+side 277
+▁new 278
+▁part 279
+▁could 280
+▁said 281
+▁those 282
+▁di 283
+▁N 284
+▁him 285
+und 286
+▁way 287
+ish 288
+▁did 289
+ving 290
+▁first 291
+ugh 292
+ha 293
+vo 294
+ture 295
+▁W 296
+tru 297
+▁make 298
+▁ro 299
+ence 300
+▁every 301
+▁app 302
+▁um 303
+▁business 304
+▁per 305
+▁even 306
+ine 307
+mb 308
+▁back 309
+▁here 310
+▁quarter 311
+▁good 312
+▁take 313
+▁through 314
+ward 315
+▁market 316
+form 317
+▁much 318
+▁need 319
+▁call 320
+▁continue 321
+▁yeah 322
+▁tra 323
+▁mean 324
+line 325
+ook 326
+▁sha 327
+▁kind 328
+▁ga 329
+ize 330
+▁little 331
+▁mu 332
+▁again 333
+▁gu 334
+▁ta 335
+▁high 336
+R 337
+▁come 338
+ph 339
+▁hu 340
+▁should 341
+O 342
+▁most 343
+▁down 344
+▁give 345
+▁talk 346
+▁Co 347
+! 348
+▁after 349
+▁great 350
+2 351
+▁let 352
+▁product 353
+▁start 354
+▁K 355
+ative 356
+▁day 357
+▁last 358
+tain 359
+▁long 360
+▁though 361
+▁something 362
+▁expect 363
+▁before 364
+▁two 365
+U 366
+▁result 367
+port 368
+▁actually 369
+▁made 370
+▁growth 371
+ship 372
+▁different 373
+▁question 374
+▁customer 375
+▁still 376
+▁under 377
+▁change 378
+3 379
+▁point 380
+5 381
+▁plan 382
+ness 383
+V 384
+▁show 385
+que 386
+▁number 387
+▁own 388
+ible 389
+qua 390
+▁help 391
+▁world 392
+▁place 393
+▁va 394
+▁invest 395
+▁second 396
+▁same 397
+▁next 398
+▁imp 399
+▁count 400
+▁life 401
+▁why 402
+▁believe 403
+▁20 404
+▁trans 405
+▁term 406
+▁feel 407
+▁might 408
+▁happen 409
+▁turn 410
+▁around 411
+▁such 412
+▁while 413
+▁strong 414
+▁try 415
+▁important 416
+▁large 417
+▁better 418
+ific 419
+▁develop 420
+▁increase 421
+▁tell 422
+▁company 423
+▁interest 424
+▁play 425
+▁another 426
+serv 427
+▁build 428
+▁person 429
+A 430
+▁name 431
+4 432
+ability 433
+▁provide 434
+press 435
+▁focus 436
+▁certain 437
+▁found 438
+▁service 439
+▁Ro 440
+▁each 441
+▁always 442
+▁system 443
+▁Re 444
+▁during 445
+▁level 446
+▁course 447
+▁support 448
+▁grow 449
+▁improve 450
+▁data 451
+▁between 452
+▁drive 453
+▁team 454
+6 455
+▁okay 456
+▁half 457
+▁impact 458
+▁maybe 459
+▁future 460
+▁understand 461
+▁process 462
+▁remain 463
+▁close 464
+▁hard 465
+▁value 466
+Y 467
+▁revenue 468
+▁program 469
+▁idea 470
+▁project 471
+▁keep 472
+▁follow 473
+▁direct 474
+▁small 475
+S 476
+▁experience 477
+▁power 478
+▁across 479
+8 480
+% 481
+7 482
+9 483
+P 484
+Q 485
+I 486
+C 487
+T 488
+E 489
+D 490
+Z 491
+M 492
+B 493
+q 494
+$ 495
+N 496
+G 497
+F 498
+X 499
+K 500
+W 501
+: 502
+& 503
+¢ 504
+/ 505
+° 506
+₽ 507
+¥ 508
+€ 509
+* 510
+£ 511