distilbert-base-uncased-finetuned-sst-2-english_07112024T125645

This model is a fine-tuned version of distilbert/distilbert-base-uncased-finetuned-sst-2-english on the MR Analysis Phase-3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5776
• F1: 0.8426
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Rate
No log	1.0	141	1.1776	0.5721	0.0000
No log	2.0	282	0.9785	0.6619	0.0000
No log	3.0	423	0.8326	0.7194	0.0000
1.1084	4.0	564	0.6920	0.7808	0.0000
1.1084	5.0	705	0.6907	0.7973	0.0000
1.1084	6.0	846	0.6107	0.8284	0.0000
1.1084	7.0	987	0.5776	0.8426	0.0000
0.4572	8.0	1128	0.6100	0.8523	0.0000
0.4572	9.0	1269	0.6279	0.8570	0.0000
0.4572	10.0	1410	0.6638	0.8587	0.0000
0.1637	11.0	1551	0.7340	0.8568	0.0000
0.1637	12.0	1692	0.7564	0.8596	7e-06
0.1637	13.0	1833	0.8077	0.8568	0.0000
0.1637	14.0	1974	0.7234	0.8667	0.0000
0.069	15.0	2115	0.7535	0.8664	3e-06
0.069	16.0	2256	0.7818	0.8659	0.0000
0.069	17.0	2397	0.8064	0.8646	0.0000
0.0376	18.0	2538	0.8203	0.8626	5e-07
0.0376	19.0	2679	0.8233	0.8629	1e-07
0.0376	20.0	2820	0.8235	0.8632	0.0

Testing Results

class	precision	recall	f1-score
change_request	0.918	0.651	0.762
discussion_participation	0.839	0.882	0.860
discussion_trigger	0.879	0.902	0.890
acknowledgement	0.847	0.920	0.882
critical	0.686	0.940	0.793
reference	0.802	0.947	0.869
-----------	----------	--------	---------
accuracy			0.828
macro avg	0.828	0.874	0.843
weighted avg	0.845	0.828	0.825

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.19.1