Text Classification
Transformers
Safetensors
roberta
Generated from Trainer
text-embeddings-inference
# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("contemmcm/a04f436e4d4e5de5b76ba7fd904c3c78")
model = AutoModelForSequenceClassification.from_pretrained("contemmcm/a04f436e4d4e5de5b76ba7fd904c3c78")Quick Links
a04f436e4d4e5de5b76ba7fd904c3c78
This model is a fine-tuned version of FacebookAI/roberta-large on the ccdv/patent-classification [abstract] dataset. It achieves the following results on the evaluation set:
- Loss: 1.3133
- Data Size: 1.0
- Epoch Runtime: 116.4855
- Accuracy: 0.6518
- F1 Macro: 0.5961
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: constant
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Data Size | Epoch Runtime | Accuracy | F1 Macro |
|---|---|---|---|---|---|---|---|
| No log | 0 | 0 | 2.1164 | 0 | 7.2057 | 0.1510 | 0.0292 |
| No log | 1 | 781 | 1.9938 | 0.0078 | 8.4663 | 0.2718 | 0.0863 |
| No log | 2 | 1562 | 1.5860 | 0.0156 | 9.4765 | 0.3872 | 0.1908 |
| No log | 3 | 2343 | 1.4555 | 0.0312 | 11.5807 | 0.4683 | 0.3026 |
| 0.0374 | 4 | 3124 | 1.3472 | 0.0625 | 15.1891 | 0.5483 | 0.4418 |
| 1.3362 | 5 | 3905 | 1.1729 | 0.125 | 21.9533 | 0.5645 | 0.5232 |
| 1.2058 | 6 | 4686 | 1.0861 | 0.25 | 36.7036 | 0.6178 | 0.5091 |
| 1.0744 | 7 | 5467 | 1.1440 | 0.5 | 63.8407 | 0.6174 | 0.5373 |
| 0.9986 | 8.0 | 6248 | 1.1125 | 1.0 | 121.1422 | 0.6314 | 0.5654 |
| 0.9276 | 9.0 | 7029 | 1.0343 | 1.0 | 123.2721 | 0.6502 | 0.5854 |
| 0.8526 | 10.0 | 7810 | 1.1100 | 1.0 | 119.2637 | 0.6482 | 0.5935 |
| 0.7286 | 11.0 | 8591 | 1.0734 | 1.0 | 118.6671 | 0.6601 | 0.6068 |
| 0.674 | 12.0 | 9372 | 1.2207 | 1.0 | 117.9967 | 0.6583 | 0.6129 |
| 0.558 | 13.0 | 10153 | 1.3133 | 1.0 | 116.4855 | 0.6518 | 0.5961 |
Framework versions
- Transformers 4.57.0
- Pytorch 2.8.0+cu128
- Datasets 4.3.0
- Tokenizers 0.22.1
- Downloads last month
- 2
Model tree for contemmcm/a04f436e4d4e5de5b76ba7fd904c3c78
Base model
FacebookAI/roberta-large
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="contemmcm/a04f436e4d4e5de5b76ba7fd904c3c78")