xlsr300m-khmer-cpt-100h

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Contrastive Loss: 86.7440
• Diversity Loss: 40.6903
• Codevector Perplexity: 43.5105
• Loss: 90.8130

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• distributed_type: multi-GPU
• num_devices: 3
• gradient_accumulation_steps: 16
• total_train_batch_size: 96
• total_eval_batch_size: 6
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.98) and epsilon=1e-06 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: polynomial
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 50000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Codevector Perplexity	Contrastive Loss	Diversity Loss	Validation Loss
125.4134	1.7766	1000	44.0776	98.3090	41.1258	102.4216
118.7395	3.5527	2000	99.3660	41.0202	43.3298	103.4681
115.985	5.3288	3000	100.6013	41.2204	43.5237	104.7233
113.3363	7.1049	4000	97.1429	40.7029	43.5608	101.2132
110.4359	8.8815	5000	98.6855	41.4481	44.0719	102.8303
108.3205	10.6576	6000	97.5672	40.9626	43.6800	101.6634
107.2761	12.4336	7000	96.3640	40.9157	43.5723	100.4556
105.1178	14.2097	8000	94.5814	40.9914	43.8926	98.6805
103.8618	15.9863	9000	92.5783	41.0288	44.0242	96.6812
103.2753	17.7624	10000	91.2246	40.9362	43.8229	95.3183
103.9451	19.5385	11000	89.6453	40.6490	43.8782	93.7102
102.5424	21.3146	12000	91.6800	40.6878	44.1377	95.7487
102.2143	23.0906	13000	92.6359	40.8752	43.6222	96.7234
102.5877	24.8673	14000	90.6878	41.0020	43.8892	94.7879
102.3885	26.6433	15000	93.2308	41.2784	44.6644	97.3586
100.7177	28.4194	16000	90.7890	40.6389	43.3034	94.8529
101.4842	30.1955	17000	91.4696	40.8514	44.3802	95.5547
100.4756	31.9721	18000	91.2408	40.8162	43.7597	95.3224
100.3903	33.7482	19000	90.2077	40.3539	43.5307	94.2431
99.3479	35.5243	20000	91.7194	40.6756	43.6408	95.7869
99.3767	37.3003	21000	91.7649	41.3250	43.8292	95.8974
100.2972	39.0764	22000	87.5791	40.2266	43.4941	91.6018
98.813	40.8530	23000	88.4026	39.8155	42.8212	92.3841
98.0505	42.6291	24000	89.3573	40.5817	43.4010	93.4155
99.4878	44.4052	25000	87.5768	40.3717	43.2696	91.6140
98.3515	46.1813	26000	90.4265	41.2541	44.2365	94.5519
98.7256	47.9579	27000	91.6725	40.7975	44.3128	95.7522
98.0115	49.7340	28000	90.1710	40.9520	44.5153	94.2662
97.4218	51.5101	29000	91.8110	41.4037	43.9065	95.9514
96.9836	53.2861	30000	89.3851	40.9188	43.8721	93.4770
96.3193	55.0622	31000	88.6909	41.0307	44.0690	92.7939
97.9497	56.8388	32000	89.4888	40.6430	44.1668	93.5531
97.1006	58.6149	33000	89.9310	41.1537	44.4426	94.0464
97.4338	60.3910	34000	86.9285	40.2896	43.6572	90.9575
97.9852	62.1671	35000	88.3171	40.5240	44.0891	92.3695
96.8339	63.9437	36000	89.4089	41.2256	44.6988	93.5314
97.1172	65.7198	37000	86.1843	40.6979	44.1257	90.2541
96.554	67.4958	38000	87.6182	40.3946	43.3430	91.6577
95.3302	69.2719	39000	89.2178	40.8582	44.0458	93.3036
94.7802	71.0480	40000	89.2814	41.2579	44.8038	93.4072
96.2427	72.8246	41000	87.0815	40.2846	43.6098	91.1100
96.099	74.6007	42000	84.4180	40.3785	43.7319	88.4559
94.8186	76.3768	43000	87.3455	40.3647	43.6168	91.3820
97.8576	78.1528	44000	89.1611	41.1056	44.0065	93.2716
95.504	79.9295	45000	87.3539	41.1623	44.2849	91.4702
96.7857	81.7055	46000	86.3707	40.4797	44.1593	90.4187
95.8492	83.4816	47000	88.2271	41.2915	44.7006	92.3562
96.2915	85.2577	48000	87.8822	40.7081	44.0356	91.9530
95.4427	87.0338	49000	88.0481	41.2303	44.4313	92.1712
95.4937	88.8104	50000	86.7440	40.6903	43.5105	90.8130

Framework versions

• Transformers 4.57.6
• Pytorch 2.10.0+cu128
• Datasets 2.19.1
• Tokenizers 0.22.2