wav2vec2-xlsr-300m-hre-v1

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the hre-audio-dataset8 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7875
• Cer Ortho: 51.8126
• Cer: 51.3707

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer Ortho	Cer
3.4833	0.4608	100	3.4517	99.0488	99.0248
3.1952	0.9217	200	3.0203	99.0488	99.0248
2.9279	1.3825	300	2.8936	99.0488	99.0248
2.8948	1.8433	400	2.7970	91.8521	91.9595
2.7727	2.3041	500	2.6553	95.4774	95.4186
2.4391	2.7650	600	2.4019	87.9397	87.6541
2.0669	3.2258	700	1.9134	83.4709	78.6017
1.7675	3.6866	800	1.7641	86.1630	78.7305
1.4705	4.1475	900	1.5949	84.8708	75.3266
1.3456	4.6083	1000	1.3056	78.8047	71.2052
1.3088	5.0691	1100	1.2253	78.9663	69.8068
1.1681	5.5300	1200	1.1573	77.6920	67.7093
0.9910	5.9908	1300	1.1332	74.0129	66.0718
0.9176	6.4516	1400	1.0091	64.1242	61.5823
1.0415	6.9124	1500	0.9127	61.3245	60.7176
0.9014	7.3733	1600	0.9495	60.3553	59.7424
0.8234	7.8341	1700	0.9731	61.4860	60.9384
0.7313	8.2949	1800	0.9365	60.0503	59.2456
0.7660	8.7558	1900	0.9118	59.8349	59.6136
0.6833	9.2166	2000	0.8462	56.5506	56.5593
0.6984	9.6774	2100	0.8768	58.0761	57.3689
0.6272	10.1382	2200	0.9320	58.5068	57.9025
0.6431	10.5991	2300	0.8695	56.4070	56.4489
0.5980	11.0599	2400	0.8595	56.4070	56.5225
0.5701	11.5207	2500	0.9245	57.3044	56.6697
0.6168	11.9816	2600	0.7961	54.0380	53.6707
0.5916	12.4424	2700	0.8187	55.2225	55.0138
0.6809	12.9032	2800	0.8054	54.8636	54.6826
0.5780	13.3641	2900	0.8034	53.9304	53.7626
0.5056	13.8249	3000	0.8062	53.9483	53.8730
0.5218	14.2857	3100	0.8272	54.3611	54.2226
0.5384	14.7465	3200	0.8102	53.5535	52.9899
0.5307	15.2074	3300	0.7969	52.8715	52.2907
0.4648	15.6682	3400	0.7977	53.2843	52.6403
0.4624	16.1290	3500	0.8211	53.5714	52.8795
0.4073	16.5899	3600	0.7983	52.4587	51.7755
0.4627	17.0507	3700	0.7947	52.9612	52.2355
0.4158	17.5115	3800	0.8145	52.1177	51.6651
0.3863	17.9724	3900	0.7885	52.1177	51.7571
0.4024	18.4332	4000	0.8038	51.9921	51.4259
0.3865	18.8940	4100	0.8207	52.0101	51.3523
0.4091	19.3548	4200	0.7871	51.7408	51.1684
0.3589	19.8157	4300	0.7872	51.8665	51.3891
0.3606	20.0	4340	0.7875	51.8126	51.3707

Framework versions

• Transformers 5.13.1
• Pytorch 2.11.0+cu128
• Datasets 2.18.0
• Tokenizers 0.22.2