Text Classification
Transformers
Safetensors
Italian
deberta-v2
Generated from Trainer
subjectivity-detection
deberta-v3
Instructions to use AIWizards/mdeberta-v3-base-subjectivity-italian with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AIWizards/mdeberta-v3-base-subjectivity-italian with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="AIWizards/mdeberta-v3-base-subjectivity-italian")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-italian") model = AutoModelForSequenceClassification.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-italian") - Notebooks
- Google Colab
- Kaggle
Improve model card: Add pipeline tag, update license, expand description and usage
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,25 +1,36 @@
|
|
| 1 |
---
|
| 2 |
-
library_name: transformers
|
| 3 |
-
license: mit
|
| 4 |
base_model: microsoft/mdeberta-v3-base
|
| 5 |
-
|
| 6 |
-
-
|
|
|
|
|
|
|
| 7 |
metrics:
|
| 8 |
- accuracy
|
| 9 |
- f1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
model-index:
|
| 11 |
- name: mdeberta-v3-base-subjectivity-italian
|
| 12 |
results: []
|
| 13 |
-
|
| 14 |
-
-
|
| 15 |
---
|
| 16 |
|
| 17 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 18 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 19 |
-
|
| 20 |
# mdeberta-v3-base-subjectivity-italian
|
| 21 |
|
| 22 |
-
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
It achieves the following results on the evaluation set:
|
| 24 |
- Loss: 0.7922
|
| 25 |
- Macro F1: 0.7490
|
|
@@ -32,15 +43,28 @@ It achieves the following results on the evaluation set:
|
|
| 32 |
|
| 33 |
## Model description
|
| 34 |
|
| 35 |
-
|
|
|
|
|
|
|
| 36 |
|
| 37 |
## Intended uses & limitations
|
| 38 |
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
## Training and evaluation data
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
| 44 |
|
| 45 |
## Training procedure
|
| 46 |
|
|
@@ -66,10 +90,49 @@ The following hyperparameters were used during training:
|
|
| 66 |
| 0.4326 | 5.0 | 505 | 0.7883 | 0.7463 | 0.7413 | 0.7522 | 0.6322 | 0.6105 | 0.6554 | 0.7976 |
|
| 67 |
| 0.4326 | 6.0 | 606 | 0.7922 | 0.7490 | 0.7409 | 0.7602 | 0.6402 | 0.6020 | 0.6836 | 0.7961 |
|
| 68 |
|
| 69 |
-
|
| 70 |
### Framework versions
|
| 71 |
|
| 72 |
- Transformers 4.49.0
|
| 73 |
- Pytorch 2.5.1+cu121
|
| 74 |
- Datasets 3.3.1
|
| 75 |
-
- Tokenizers 0.21.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
base_model: microsoft/mdeberta-v3-base
|
| 3 |
+
language:
|
| 4 |
+
- it
|
| 5 |
+
library_name: transformers
|
| 6 |
+
license: cc-by-4.0
|
| 7 |
metrics:
|
| 8 |
- accuracy
|
| 9 |
- f1
|
| 10 |
+
tags:
|
| 11 |
+
- generated_from_trainer
|
| 12 |
+
- text-classification
|
| 13 |
+
- subjectivity-detection
|
| 14 |
+
- deberta-v3
|
| 15 |
+
pipeline_tag: text-classification
|
| 16 |
+
paper: 2507.11764
|
| 17 |
+
repo_url: https://github.com/MatteoFasulo/clef2025-checkthat
|
| 18 |
model-index:
|
| 19 |
- name: mdeberta-v3-base-subjectivity-italian
|
| 20 |
results: []
|
| 21 |
+
datasets:
|
| 22 |
+
- MatteoFasulo/clef2025_checkthat_task1_subjectivity
|
| 23 |
---
|
| 24 |
|
|
|
|
|
|
|
|
|
|
| 25 |
# mdeberta-v3-base-subjectivity-italian
|
| 26 |
|
| 27 |
+
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) for **Subjectivity Detection in News Articles**. It was developed by AI Wizards as part of their participation in the **CLEF 2025 CheckThat! Lab Task 1**.
|
| 28 |
+
|
| 29 |
+
The model aims to classify sentences as subjective (opinion-laden) or objective. Its primary strategy involves enhancing transformer-based classifiers by integrating sentiment scores, derived from an auxiliary model, with sentence representations. This approach has been shown to significantly boost performance, especially the subjective F1 score.
|
| 30 |
+
|
| 31 |
+
For more details, refer to the paper: [AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles](https://arxiv.org/abs/2507.11764).
|
| 32 |
+
The official code repository is available at: [https://github.com/MatteoFasulo/clef2025-checkthat](https://github.com/MatteoFasulo/clef2025-checkthat).
|
| 33 |
+
|
| 34 |
It achieves the following results on the evaluation set:
|
| 35 |
- Loss: 0.7922
|
| 36 |
- Macro F1: 0.7490
|
|
|
|
| 43 |
|
| 44 |
## Model description
|
| 45 |
|
| 46 |
+
This `mdeberta-v3-base-subjectivity-italian` model is a transformer-based classifier specifically designed for subjectivity detection in news articles. It distinguishes between subjective (opinion-laden) and objective sentences. The model's innovation lies in augmenting transformer embeddings with sentiment signals from an auxiliary model, leading to consistent performance gains, particularly in the subjective F1 score. It also incorporates robust decision threshold calibration to counter class imbalances prevalent across different languages.
|
| 47 |
+
|
| 48 |
+
This model was evaluated across monolingual settings (Arabic, German, English, Italian, Bulgarian), zero-shot transfer (Greek, Polish, Romanian, Ukrainian), and multilingual training, demonstrating strong generalization capabilities.
|
| 49 |
|
| 50 |
## Intended uses & limitations
|
| 51 |
|
| 52 |
+
**Intended Uses:**
|
| 53 |
+
- **Subjectivity Detection**: Classifying sentences in news articles as subjective or objective.
|
| 54 |
+
- **Fact-Checking Pipelines**: Serving as a component to identify opinionated content that might require further scrutiny.
|
| 55 |
+
- **Journalism Support**: Aiding journalists in analyzing content for bias or sentiment.
|
| 56 |
+
- **Misinformation Combatting**: Contributing to systems designed to detect and combat misinformation by flagging subjective claims.
|
| 57 |
+
|
| 58 |
+
**Limitations:**
|
| 59 |
+
- **Class Imbalance Sensitivity**: While decision threshold calibration was applied, the model's performance can be sensitive to the class distribution of the evaluation data. An initial submission error during the CLEF 2025 challenge illustrated this sensitivity.
|
| 60 |
+
- **Domain Specificity**: Optimized for news articles; performance might vary on text from significantly different domains.
|
| 61 |
+
- **Sentiment Model Dependency**: The effectiveness of sentiment augmentation depends on the quality and relevance of the auxiliary sentiment model used.
|
| 62 |
|
| 63 |
## Training and evaluation data
|
| 64 |
|
| 65 |
+
This model was fine-tuned on data from the **CLEF 2025 CheckThat! Lab Task 1: Subjectivity Detection in News Articles**. The training and development datasets were provided for Arabic, German, English, Italian, and Bulgarian. For the final evaluation, additional unseen languages such as Greek, Romanian, Polish, and Ukrainian were included to assess the model's generalization capabilities.
|
| 66 |
+
|
| 67 |
+
The training process specifically addressed class imbalance, which was a notable characteristic across these languages, by employing decision threshold calibration optimized on the development set.
|
| 68 |
|
| 69 |
## Training procedure
|
| 70 |
|
|
|
|
| 90 |
| 0.4326 | 5.0 | 505 | 0.7883 | 0.7463 | 0.7413 | 0.7522 | 0.6322 | 0.6105 | 0.6554 | 0.7976 |
|
| 91 |
| 0.4326 | 6.0 | 606 | 0.7922 | 0.7490 | 0.7409 | 0.7602 | 0.6402 | 0.6020 | 0.6836 | 0.7961 |
|
| 92 |
|
|
|
|
| 93 |
### Framework versions
|
| 94 |
|
| 95 |
- Transformers 4.49.0
|
| 96 |
- Pytorch 2.5.1+cu121
|
| 97 |
- Datasets 3.3.1
|
| 98 |
+
- Tokenizers 0.21.0
|
| 99 |
+
|
| 100 |
+
## How to use
|
| 101 |
+
|
| 102 |
+
You can use this model for text classification (subjectivity detection) with the `transformers` library:
|
| 103 |
+
|
| 104 |
+
```python
|
| 105 |
+
from transformers import pipeline
|
| 106 |
+
|
| 107 |
+
# Load the text classification pipeline
|
| 108 |
+
classifier = pipeline(
|
| 109 |
+
"text-classification",
|
| 110 |
+
model="MatteoFasulo/mdeberta-v3-base-subjectivity-italian",
|
| 111 |
+
tokenizer="microsoft/mdeberta-v3-base",
|
| 112 |
+
)
|
| 113 |
+
|
| 114 |
+
# Example usage:
|
| 115 |
+
# A subjective sentence
|
| 116 |
+
result_subj = classifier("Questa è una scoperta affascinante e fantastica!")
|
| 117 |
+
print(f"Sentence: 'This is a truly amazing and groundbreaking discovery!' -> {result_subj}")
|
| 118 |
+
|
| 119 |
+
# An objective sentence
|
| 120 |
+
result_obj = classifier("In particolare Volkswagen e Stellantis son o arrivati a cedere il 7% in Borsa.")
|
| 121 |
+
print(f"Sentence: 'The new policy will be implemented next quarter.' -> {result_obj}")
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
+
## Citation
|
| 125 |
+
|
| 126 |
+
If you find our work helpful or inspiring, please feel free to cite it:
|
| 127 |
+
|
| 128 |
+
```bibtex
|
| 129 |
+
@misc{fasulo2025aiwizardscheckthat2025,
|
| 130 |
+
title={AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles},
|
| 131 |
+
author={Matteo Fasulo and Luca Babboni and Luca Tedeschini},
|
| 132 |
+
year={2025},
|
| 133 |
+
eprint={2507.11764},
|
| 134 |
+
archivePrefix={arXiv},
|
| 135 |
+
primaryClass={cs.CL},
|
| 136 |
+
url={https://arxiv.org/abs/2507.11764},
|
| 137 |
+
}
|
| 138 |
+
```
|