Aditya109 commited on
Commit
d39a21b
·
verified ·
1 Parent(s): eb75787

Replace cited large-v3 baseline with measured 65.93 WER / 37.68 CER

Browse files
Files changed (1) hide show
  1. README.md +3 -5
README.md CHANGED
@@ -49,18 +49,16 @@ Read that one if you want to know how these numbers were produced.
49
 
50
  ## Results
51
 
52
- Official Vaani Bhojpuri test split, n = 1,426. All rows measured on the same
53
- utterances with the same normalizer, except where noted.
54
 
55
  | Model | Params | WER ↓ | CER ↓ |
56
  |---|---|---|---|
57
  | `openai/whisper-small` (untrained) | 244M | 122.07 | 78.75 |
58
- | `openai/whisper-large-v3` (untrained) | 1.55B | 66.06 | 37.71 |
59
  | `ARTPARK-IISc/SraVaani` | 430M | **34.80** | 20.29 |
60
  | **This model** | 244M | 36.41 | **17.05** |
61
 
62
- † Cited from the project spec, not measured here.
63
-
64
  Against SraVaani — the model trained by the dataset's own authors — this is a
65
  split decision, and both halves are statistically significant under a paired
66
  bootstrap: SraVaani is better on WER by 1.61 (95% CI [+0.83, +2.44]), this
 
49
 
50
  ## Results
51
 
52
+ Official Vaani Bhojpuri test split, n = 1,426. Every row measured on the same
53
+ utterances, with the same normalizer and the same metric implementation.
54
 
55
  | Model | Params | WER ↓ | CER ↓ |
56
  |---|---|---|---|
57
  | `openai/whisper-small` (untrained) | 244M | 122.07 | 78.75 |
58
+ | `openai/whisper-large-v3` (untrained) | 1.55B | 65.93 | 37.68 |
59
  | `ARTPARK-IISc/SraVaani` | 430M | **34.80** | 20.29 |
60
  | **This model** | 244M | 36.41 | **17.05** |
61
 
 
 
62
  Against SraVaani — the model trained by the dataset's own authors — this is a
63
  split decision, and both halves are statistically significant under a paired
64
  bootstrap: SraVaani is better on WER by 1.61 (95% CI [+0.83, +2.44]), this