w2vbert-lingala-waxal

This model is a fine-tuned version of sulaimank/w2v-bert-lingala-109h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1669
• Wer: 0.0961
• Cer: 0.0600
• Zindi: 0.9219

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 60.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer	Zindi
0.6526	0.6540	500	0.1291	0.1357	0.0695	0.8974
0.5744	1.3074	1000	0.1299	0.1408	0.0750	0.8921
0.5442	1.9614	1500	0.1134	0.1322	0.0688	0.8995
0.5000	2.6148	2000	0.1197	0.1350	0.0683	0.8983
0.4402	3.2681	2500	0.1260	0.1352	0.0700	0.8974
0.4074	3.9222	3000	0.1208	0.1296	0.0687	0.9009
0.4248	4.5755	3500	0.1269	0.1233	0.0654	0.9056
0.3575	5.2289	4000	0.1309	0.1269	0.0683	0.9024
0.3851	5.8829	4500	0.1367	0.1224	0.0657	0.9060
0.3517	6.5363	5000	0.1302	0.1243	0.0639	0.9059
0.3091	7.1897	5500	0.1356	0.1175	0.0626	0.9100
0.3057	7.8437	6000	0.1183	0.1172	0.0620	0.9104
0.2593	8.4971	6500	0.1410	0.1209	0.0645	0.9073
0.2649	9.1504	7000	0.1303	0.1138	0.0636	0.9113
0.2522	9.8044	7500	0.1299	0.1147	0.0624	0.9115
0.2238	10.4578	8000	0.1396	0.1149	0.0617	0.9117
0.1899	11.1112	8500	0.1432	0.1100	0.0618	0.9141
0.2092	11.7652	9000	0.1362	0.1081	0.0635	0.9142
0.1909	12.4186	9500	0.1392	0.1079	0.0631	0.9145
0.1500	13.0719	10000	0.1488	0.1044	0.0609	0.9173
0.1639	13.7260	10500	0.1558	0.1014	0.0612	0.9187
0.1249	14.3793	11000	0.1568	0.1004	0.0609	0.9194
0.1179	15.0327	11500	0.1500	0.1023	0.0609	0.9184
0.1165	15.6867	12000	0.1552	0.1004	0.0608	0.9194
0.1014	16.3401	12500	0.1587	0.0989	0.0587	0.9212
0.1058	16.9941	13000	0.1652	0.0959	0.0596	0.9223
0.0829	17.6475	13500	0.1596	0.0991	0.0603	0.9203
0.0706	18.3009	14000	0.1731	0.0988	0.0589	0.9211
0.0753	18.9549	14500	0.1704	0.0964	0.0603	0.9216
0.0877	19.6082	15000	0.1551	0.0984	0.0613	0.9201
0.0513	20.2616	15500	0.1645	0.0978	0.0588	0.9217
0.0560	20.9156	16000	0.1596	0.0974	0.0581	0.9222
0.0577	21.5690	16500	0.1561	0.0984	0.0624	0.9196
0.0322	22.2224	17000	0.1680	0.0961	0.0605	0.9217
0.0507	22.8764	17500	0.1708	0.0968	0.0595	0.9219
0.0388	23.5298	18000	0.1669	0.0961	0.0600	0.9219

Framework versions

• Transformers 5.13.0
• Pytorch 2.12.1+cu130
• Datasets 3.6.0
• Tokenizers 0.22.2