nielsr HF Staff commited on
Commit
5e585b4
·
verified ·
1 Parent(s): daa7ad2

Improve model card: Add pipeline tag, update license, GitHub link, and usage example

Browse files

This PR significantly improves the model card for `mdeberta-v3-base-subjectivity-sentiment-italian` by:

* **Adding `pipeline_tag: text-classification`** to ensure the model is easily discoverable for its intended task.
* **Updating the `license` to `cc-by-4.0`**, correcting the previous `mit` license as specified in the associated GitHub repository.
* **Including a direct link to the GitHub repository** (`https://github.com/MatteoFasulo/clef2025-checkthat`) for easy access to the source code and project materials.
* **Providing a `transformers` pipeline usage example** in a new "How to use" section, simplifying model inference for users.
* **Adding more specific `tags`**: `deberta-v2`, `subjectivity-detection`, and `sentiment-analysis` for more accurate categorization.
* **Populating the "Model description" and "Intended uses & limitations" sections** with information extracted from the paper abstract and the project's GitHub README.
* **Correcting the formatting of the arXiv paper link** in the model card content to ensure it is clickable.

These enhancements contribute to a more comprehensive, accurate, and user-friendly model card on the Hugging Face Hub.

Files changed (1) hide show
  1. README.md +48 -23
README.md CHANGED
@@ -1,26 +1,29 @@
1
  ---
 
 
 
 
2
  library_name: transformers
3
- tags:
4
- - generated_from_trainer
5
  metrics:
6
- - accuracy
7
- - f1
 
 
 
 
 
 
8
  model-index:
9
- - name: mdeberta-v3-base-subjectivity-sentiment-italian
10
- results: []
11
- license: mit
12
- language:
13
- - it
14
- base_model:
15
- - microsoft/mdeberta-v3-base
16
  ---
17
 
18
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
19
- should probably proofread and complete it, then remove this comment. -->
20
-
21
  # mdeberta-v3-base-subjectivity-sentiment-italian
22
 
23
- This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](arxiv.org/abs/2507.11764).
 
 
24
  It achieves the following results on the evaluation set:
25
  - Loss: 0.6602
26
  - Macro F1: 0.7437
@@ -33,16 +36,38 @@ It achieves the following results on the evaluation set:
33
 
34
  ## Model description
35
 
36
- More information needed
37
 
38
  ## Intended uses & limitations
39
 
40
- More information needed
41
 
42
  ## Training and evaluation data
43
 
44
  More information needed
45
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  ## Training procedure
47
 
48
  ### Training hyperparameters
@@ -59,12 +84,12 @@ The following hyperparameters were used during training:
59
  ### Training results
60
 
61
  | Training Loss | Epoch | Step | Validation Loss | Macro F1 | Macro P | Macro R | Subj F1 | Subj P | Subj R | Accuracy |
62
- |:-------------:|:-----:|:----:|:---------------:|:--------:|:-------:|:-------:|:-------:|:------:|:------:|:--------:|
63
- | No log | 1.0 | 101 | 0.6392 | 0.7244 | 0.7284 | 0.7208 | 0.5913 | 0.6071 | 0.5763 | 0.7886 |
64
- | No log | 2.0 | 202 | 0.5375 | 0.6731 | 0.7018 | 0.7579 | 0.6064 | 0.4548 | 0.9096 | 0.6867 |
65
- | No log | 3.0 | 303 | 0.5731 | 0.7453 | 0.7373 | 0.7563 | 0.6349 | 0.5970 | 0.6780 | 0.7931 |
66
- | No log | 4.0 | 404 | 0.5788 | 0.7522 | 0.7405 | 0.7752 | 0.6534 | 0.5848 | 0.7401 | 0.7916 |
67
- | 0.4395 | 5.0 | 505 | 0.6922 | 0.7491 | 0.7400 | 0.7628 | 0.6423 | 0.5971 | 0.6949 | 0.7946 |
68
  | 0.4395 | 6.0 | 606 | 0.6602 | 0.7437 | 0.7322 | 0.7690 | 0.6437 | 0.5696 | 0.7401 | 0.7826 |
69
 
70
 
 
1
  ---
2
+ base_model:
3
+ - microsoft/mdeberta-v3-base
4
+ language:
5
+ - it
6
  library_name: transformers
7
+ license: cc-by-4.0
 
8
  metrics:
9
+ - accuracy
10
+ - f1
11
+ tags:
12
+ - generated_from_trainer
13
+ - deberta-v2
14
+ - subjectivity-detection
15
+ - sentiment-analysis
16
+ pipeline_tag: text-classification
17
  model-index:
18
+ - name: mdeberta-v3-base-subjectivity-sentiment-italian
19
+ results: []
 
 
 
 
 
20
  ---
21
 
 
 
 
22
  # mdeberta-v3-base-subjectivity-sentiment-italian
23
 
24
+ This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](https://arxiv.org/abs/2507.11764).
25
+ For the official code and materials, please refer to the [GitHub repository](https://github.com/MatteoFasulo/clef2025-checkthat).
26
+
27
  It achieves the following results on the evaluation set:
28
  - Loss: 0.6602
29
  - Macro F1: 0.7437
 
36
 
37
  ## Model description
38
 
39
+ This model, `mdeberta-v3-base-subjectivity-sentiment-italian`, is part of the AI Wizards' submission to the CLEF 2025 CheckThat! Lab Task 1: Subjectivity Detection in News Articles. Its primary goal is to classify sentences as subjective or objective. A key innovation in its development involved enhancing transformer-based classifiers, specifically `mDeBERTaV3-base`, by integrating sentiment scores derived from an auxiliary model with sentence representations. This sentiment-augmented architecture, combined with decision threshold calibration to address class imbalance, consistently boosted performance, especially the subjective F1 score.
40
 
41
  ## Intended uses & limitations
42
 
43
+ This model is intended for identifying whether a sentence is subjective (opinion-laden) or objective in news articles, which is crucial for combating misinformation, improving fact-checking pipelines, and supporting journalists. It has been evaluated in monolingual (Italian, Arabic, German, English, Bulgarian), zero-shot (Greek, Polish, Romanian, Ukrainian), and multilingual settings. While the sentiment augmentation consistently improved performance, users should be aware that the model's effectiveness may vary across languages and specific domains not covered in the training data. The model was trained with specific hyperparameters and decision threshold calibration for class imbalance, which are critical for its reported performance.
44
 
45
  ## Training and evaluation data
46
 
47
  More information needed
48
 
49
+ ## How to use
50
+
51
+ You can use this model with the `transformers` library for text classification:
52
+
53
+ ```python
54
+ from transformers import pipeline
55
+
56
+ classifier = pipeline(
57
+ "text-classification",
58
+ model="MatteoFasulo/mdeberta-v3-base-subjectivity-sentiment-italian"
59
+ )
60
+
61
+ text_objective = "The capital of France is Paris."
62
+ text_subjective = "This movie is absolutely brilliant and a must-watch!"
63
+
64
+ print(classifier(text_objective))
65
+ # Expected output: [{'label': 'OBJ', 'score': 0.99...}] (scores may vary)
66
+
67
+ print(classifier(text_subjective))
68
+ # Expected output: [{'label': 'SUBJ', 'score': 0.98...}] (scores may vary)
69
+ ```
70
+
71
  ## Training procedure
72
 
73
  ### Training hyperparameters
 
84
  ### Training results
85
 
86
  | Training Loss | Epoch | Step | Validation Loss | Macro F1 | Macro P | Macro R | Subj F1 | Subj P | Subj R | Accuracy |
87
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:-------:|:-------:|:-------:|:------:|:------:|:--------:|\
88
+ | No log | 1.0 | 101 | 0.6392 | 0.7244 | 0.7284 | 0.7208 | 0.5913 | 0.6071 | 0.5763 | 0.7886 |\
89
+ | No log | 2.0 | 202 | 0.5375 | 0.6731 | 0.7018 | 0.7579 | 0.6064 | 0.4548 | 0.9096 | 0.6867 |\
90
+ | No log | 3.0 | 303 | 0.5731 | 0.7453 | 0.7373 | 0.7563 | 0.6349 | 0.5970 | 0.6780 | 0.7931 |\
91
+ | No log | 4.0 | 404 | 0.5788 | 0.7522 | 0.7405 | 0.7752 | 0.6534 | 0.5848 | 0.7401 | 0.7916 |\
92
+ | 0.4395 | 5.0 | 505 | 0.6922 | 0.7491 | 0.7400 | 0.7628 | 0.6423 | 0.5971 | 0.6949 | 0.7946 |\
93
  | 0.4395 | 6.0 | 606 | 0.6602 | 0.7437 | 0.7322 | 0.7690 | 0.6437 | 0.5696 | 0.7401 | 0.7826 |
94
 
95