Morocco-Darija-STT-large-v1.4

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5198
• Wer: 0.8556

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 10
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.7653	0.3067	25	0.3849	1.0818
2.508	0.6135	50	0.2826	0.7449
2.3273	0.9202	75	0.2505	0.9422
1.8365	1.2209	100	0.2530	0.8002
1.8384	1.5276	125	0.2426	0.9928
1.7393	1.8344	150	0.2465	1.1167
1.3569	2.1350	175	0.2436	1.1889
1.4166	2.4417	200	0.2577	1.1083
1.4106	2.7485	225	0.2524	0.8159
1.3606	3.0491	250	0.2627	0.8532
1.0839	3.3558	275	0.2638	0.8363
1.1326	3.6626	300	0.2675	1.2395
1.1912	3.9693	325	0.2765	0.9374
0.8516	4.2699	350	0.2932	0.8195
0.9524	4.5767	375	0.2891	1.0830
0.9468	4.8834	400	0.2875	1.1107
0.7368	5.1840	425	0.2983	1.0566
0.7044	5.4908	450	0.3088	1.0084
0.7737	5.7975	475	0.3218	0.9952
0.5939	6.0982	500	0.3258	1.1613
0.5475	6.4049	525	0.3347	0.9495
0.4961	6.7117	550	0.3347	0.9615
0.6437	7.0123	575	0.3393	0.9458
0.3652	7.3190	600	0.3598	0.9290
0.3831	7.6258	625	0.3642	1.2142
0.4319	7.9325	650	0.3649	1.0048
0.2324	8.2331	675	0.3708	1.0108
0.2729	8.5399	700	0.3823	0.9483
0.2916	8.8466	725	0.3825	1.2250
0.1634	9.1472	750	0.3994	0.9182
0.1777	9.4540	775	0.3858	0.9795
0.1939	9.7607	800	0.3995	1.0048
0.1885	10.0613	825	0.4005	0.9723
0.1186	10.3681	850	0.4217	0.9783
0.1073	10.6748	875	0.4219	0.9783
0.1286	10.9816	900	0.4161	0.9073
0.0747	11.2822	925	0.4312	0.9579
0.0786	11.5890	950	0.4350	0.9458
0.0781	11.8957	975	0.4344	1.2298
0.0528	12.1963	1000	0.4414	0.9651
0.0551	12.5031	1025	0.4299	0.9832
0.0673	12.8098	1050	0.4316	0.9302
0.052	13.1104	1075	0.4393	0.9134
0.0488	13.4172	1100	0.4402	0.9073
0.0539	13.7239	1125	0.4458	0.9134
0.0424	14.0245	1150	0.4434	0.9675
0.0398	14.3313	1175	0.4423	0.9446
0.0485	14.6380	1200	0.4346	0.8941
0.0452	14.9448	1225	0.4670	0.8953
0.0353	15.2454	1250	0.4638	0.9085
0.0382	15.5521	1275	0.4703	0.9446
0.0405	15.8589	1300	0.4534	0.9242
0.0299	16.1595	1325	0.4604	0.8700
0.0312	16.4663	1350	0.4486	0.9170
0.0406	16.7730	1375	0.4791	0.9471
0.0285	17.0736	1400	0.4646	0.8869
0.029	17.3804	1425	0.4541	0.9097
0.0297	17.6871	1450	0.4712	0.9073
0.0323	17.9939	1475	0.4647	0.9110
0.0231	18.2945	1500	0.4664	0.8688
0.0237	18.6012	1525	0.4637	0.9422
0.0284	18.9080	1550	0.4699	0.8556
0.0229	19.2086	1575	0.4686	0.8664
0.0226	19.5153	1600	0.4749	1.1348
0.024	19.8221	1625	0.4733	1.1468
0.0173	20.1227	1650	0.4956	1.1408
0.0211	20.4294	1675	0.4859	1.1360
0.0198	20.7362	1700	0.4898	0.8712
0.022	21.0368	1725	0.4818	0.8845
0.0158	21.3436	1750	0.4963	0.8833
0.0169	21.6503	1775	0.4999	0.8544
0.0202	21.9571	1800	0.4862	0.8400
0.0135	22.2577	1825	0.5005	0.8267
0.016	22.5644	1850	0.5049	0.8520
0.0169	22.8712	1875	0.5053	0.8484
0.0122	23.1718	1900	0.5008	0.8508
0.0116	23.4785	1925	0.5091	0.8700
0.0147	23.7853	1950	0.5013	0.8592
0.0155	24.0859	1975	0.5074	0.8448
0.012	24.3926	2000	0.5088	0.8484
0.0138	24.6994	2025	0.5151	0.8580
0.0146	25.0	2050	0.5110	0.8616
0.0102	25.3067	2075	0.5118	0.8628
0.0118	25.6135	2100	0.5181	0.8460
0.0128	25.9202	2125	0.5161	0.8412
0.011	26.2209	2150	0.5118	0.8339
0.0103	26.5276	2175	0.5176	0.8387
0.0132	26.8344	2200	0.5175	0.8484
0.0103	27.1350	2225	0.5163	0.8592
0.0106	27.4417	2250	0.5185	0.8472
0.0105	27.7485	2275	0.5162	0.8532
0.0093	28.0491	2300	0.5163	0.8484
0.0094	28.3558	2325	0.5176	0.8592
0.0101	28.6626	2350	0.5188	0.8532
0.0114	28.9693	2375	0.5190	0.8496
0.0102	29.2699	2400	0.5193	0.8448
0.0088	29.5767	2425	0.5198	0.8556

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.21.0