Instructions to use AIWizards/mdeberta-v3-base-subjectivity-sentiment-italian with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AIWizards/mdeberta-v3-base-subjectivity-sentiment-italian with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="AIWizards/mdeberta-v3-base-subjectivity-sentiment-italian")# Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-sentiment-italian") model = AutoModel.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-sentiment-italian") - Notebooks
- Google Colab
- Kaggle
Improve model card: Add pipeline tag, update license, GitHub link, and usage example
Browse filesThis PR significantly improves the model card for `mdeberta-v3-base-subjectivity-sentiment-italian` by:
* **Adding `pipeline_tag: text-classification`** to ensure the model is easily discoverable for its intended task.
* **Updating the `license` to `cc-by-4.0`**, correcting the previous `mit` license as specified in the associated GitHub repository.
* **Including a direct link to the GitHub repository** (`https://github.com/MatteoFasulo/clef2025-checkthat`) for easy access to the source code and project materials.
* **Providing a `transformers` pipeline usage example** in a new "How to use" section, simplifying model inference for users.
* **Adding more specific `tags`**: `deberta-v2`, `subjectivity-detection`, and `sentiment-analysis` for more accurate categorization.
* **Populating the "Model description" and "Intended uses & limitations" sections** with information extracted from the paper abstract and the project's GitHub README.
* **Correcting the formatting of the arXiv paper link** in the model card content to ensure it is clickable.
These enhancements contribute to a more comprehensive, accurate, and user-friendly model card on the Hugging Face Hub.
|
@@ -1,26 +1,29 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
library_name: transformers
|
| 3 |
-
|
| 4 |
-
- generated_from_trainer
|
| 5 |
metrics:
|
| 6 |
-
- accuracy
|
| 7 |
-
- f1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
model-index:
|
| 9 |
-
- name: mdeberta-v3-base-subjectivity-sentiment-italian
|
| 10 |
-
|
| 11 |
-
license: mit
|
| 12 |
-
language:
|
| 13 |
-
- it
|
| 14 |
-
base_model:
|
| 15 |
-
- microsoft/mdeberta-v3-base
|
| 16 |
---
|
| 17 |
|
| 18 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 19 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 20 |
-
|
| 21 |
# mdeberta-v3-base-subjectivity-sentiment-italian
|
| 22 |
|
| 23 |
-
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](arxiv.org/abs/2507.11764).
|
|
|
|
|
|
|
| 24 |
It achieves the following results on the evaluation set:
|
| 25 |
- Loss: 0.6602
|
| 26 |
- Macro F1: 0.7437
|
|
@@ -33,16 +36,38 @@ It achieves the following results on the evaluation set:
|
|
| 33 |
|
| 34 |
## Model description
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
## Intended uses & limitations
|
| 39 |
|
| 40 |
-
|
| 41 |
|
| 42 |
## Training and evaluation data
|
| 43 |
|
| 44 |
More information needed
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
## Training procedure
|
| 47 |
|
| 48 |
### Training hyperparameters
|
|
@@ -59,12 +84,12 @@ The following hyperparameters were used during training:
|
|
| 59 |
### Training results
|
| 60 |
|
| 61 |
| Training Loss | Epoch | Step | Validation Loss | Macro F1 | Macro P | Macro R | Subj F1 | Subj P | Subj R | Accuracy |
|
| 62 |
-
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-------:|:-------:|:-------:|:------:|:------:|:--------:|
|
| 63 |
-
| No log | 1.0 | 101 | 0.6392 | 0.7244 | 0.7284 | 0.7208 | 0.5913 | 0.6071 | 0.5763 | 0.7886 |
|
| 64 |
-
| No log | 2.0 | 202 | 0.5375 | 0.6731 | 0.7018 | 0.7579 | 0.6064 | 0.4548 | 0.9096 | 0.6867 |
|
| 65 |
-
| No log | 3.0 | 303 | 0.5731 | 0.7453 | 0.7373 | 0.7563 | 0.6349 | 0.5970 | 0.6780 | 0.7931 |
|
| 66 |
-
| No log | 4.0 | 404 | 0.5788 | 0.7522 | 0.7405 | 0.7752 | 0.6534 | 0.5848 | 0.7401 | 0.7916 |
|
| 67 |
-
| 0.4395 | 5.0 | 505 | 0.6922 | 0.7491 | 0.7400 | 0.7628 | 0.6423 | 0.5971 | 0.6949 | 0.7946 |
|
| 68 |
| 0.4395 | 6.0 | 606 | 0.6602 | 0.7437 | 0.7322 | 0.7690 | 0.6437 | 0.5696 | 0.7401 | 0.7826 |
|
| 69 |
|
| 70 |
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- microsoft/mdeberta-v3-base
|
| 4 |
+
language:
|
| 5 |
+
- it
|
| 6 |
library_name: transformers
|
| 7 |
+
license: cc-by-4.0
|
|
|
|
| 8 |
metrics:
|
| 9 |
+
- accuracy
|
| 10 |
+
- f1
|
| 11 |
+
tags:
|
| 12 |
+
- generated_from_trainer
|
| 13 |
+
- deberta-v2
|
| 14 |
+
- subjectivity-detection
|
| 15 |
+
- sentiment-analysis
|
| 16 |
+
pipeline_tag: text-classification
|
| 17 |
model-index:
|
| 18 |
+
- name: mdeberta-v3-base-subjectivity-sentiment-italian
|
| 19 |
+
results: []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
---
|
| 21 |
|
|
|
|
|
|
|
|
|
|
| 22 |
# mdeberta-v3-base-subjectivity-sentiment-italian
|
| 23 |
|
| 24 |
+
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](https://arxiv.org/abs/2507.11764).
|
| 25 |
+
For the official code and materials, please refer to the [GitHub repository](https://github.com/MatteoFasulo/clef2025-checkthat).
|
| 26 |
+
|
| 27 |
It achieves the following results on the evaluation set:
|
| 28 |
- Loss: 0.6602
|
| 29 |
- Macro F1: 0.7437
|
|
|
|
| 36 |
|
| 37 |
## Model description
|
| 38 |
|
| 39 |
+
This model, `mdeberta-v3-base-subjectivity-sentiment-italian`, is part of the AI Wizards' submission to the CLEF 2025 CheckThat! Lab Task 1: Subjectivity Detection in News Articles. Its primary goal is to classify sentences as subjective or objective. A key innovation in its development involved enhancing transformer-based classifiers, specifically `mDeBERTaV3-base`, by integrating sentiment scores derived from an auxiliary model with sentence representations. This sentiment-augmented architecture, combined with decision threshold calibration to address class imbalance, consistently boosted performance, especially the subjective F1 score.
|
| 40 |
|
| 41 |
## Intended uses & limitations
|
| 42 |
|
| 43 |
+
This model is intended for identifying whether a sentence is subjective (opinion-laden) or objective in news articles, which is crucial for combating misinformation, improving fact-checking pipelines, and supporting journalists. It has been evaluated in monolingual (Italian, Arabic, German, English, Bulgarian), zero-shot (Greek, Polish, Romanian, Ukrainian), and multilingual settings. While the sentiment augmentation consistently improved performance, users should be aware that the model's effectiveness may vary across languages and specific domains not covered in the training data. The model was trained with specific hyperparameters and decision threshold calibration for class imbalance, which are critical for its reported performance.
|
| 44 |
|
| 45 |
## Training and evaluation data
|
| 46 |
|
| 47 |
More information needed
|
| 48 |
|
| 49 |
+
## How to use
|
| 50 |
+
|
| 51 |
+
You can use this model with the `transformers` library for text classification:
|
| 52 |
+
|
| 53 |
+
```python
|
| 54 |
+
from transformers import pipeline
|
| 55 |
+
|
| 56 |
+
classifier = pipeline(
|
| 57 |
+
"text-classification",
|
| 58 |
+
model="MatteoFasulo/mdeberta-v3-base-subjectivity-sentiment-italian"
|
| 59 |
+
)
|
| 60 |
+
|
| 61 |
+
text_objective = "The capital of France is Paris."
|
| 62 |
+
text_subjective = "This movie is absolutely brilliant and a must-watch!"
|
| 63 |
+
|
| 64 |
+
print(classifier(text_objective))
|
| 65 |
+
# Expected output: [{'label': 'OBJ', 'score': 0.99...}] (scores may vary)
|
| 66 |
+
|
| 67 |
+
print(classifier(text_subjective))
|
| 68 |
+
# Expected output: [{'label': 'SUBJ', 'score': 0.98...}] (scores may vary)
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
## Training procedure
|
| 72 |
|
| 73 |
### Training hyperparameters
|
|
|
|
| 84 |
### Training results
|
| 85 |
|
| 86 |
| Training Loss | Epoch | Step | Validation Loss | Macro F1 | Macro P | Macro R | Subj F1 | Subj P | Subj R | Accuracy |
|
| 87 |
+
|:-------------:|:-----:|:----:|:---------------:|:--------:|:-------:|:-------:|:-------:|:------:|:------:|:--------:|\
|
| 88 |
+
| No log | 1.0 | 101 | 0.6392 | 0.7244 | 0.7284 | 0.7208 | 0.5913 | 0.6071 | 0.5763 | 0.7886 |\
|
| 89 |
+
| No log | 2.0 | 202 | 0.5375 | 0.6731 | 0.7018 | 0.7579 | 0.6064 | 0.4548 | 0.9096 | 0.6867 |\
|
| 90 |
+
| No log | 3.0 | 303 | 0.5731 | 0.7453 | 0.7373 | 0.7563 | 0.6349 | 0.5970 | 0.6780 | 0.7931 |\
|
| 91 |
+
| No log | 4.0 | 404 | 0.5788 | 0.7522 | 0.7405 | 0.7752 | 0.6534 | 0.5848 | 0.7401 | 0.7916 |\
|
| 92 |
+
| 0.4395 | 5.0 | 505 | 0.6922 | 0.7491 | 0.7400 | 0.7628 | 0.6423 | 0.5971 | 0.6949 | 0.7946 |\
|
| 93 |
| 0.4395 | 6.0 | 606 | 0.6602 | 0.7437 | 0.7322 | 0.7690 | 0.6437 | 0.5696 | 0.7401 | 0.7826 |
|
| 94 |
|
| 95 |
|