Instructions to use AIWizards/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AIWizards/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="AIWizards/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic")# Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic") model = AutoModel.from_pretrained("AIWizards/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic") - Notebooks
- Google Colab
- Kaggle
Improve model card: Update license, add languages, expand sections, and include usage example
Browse filesThis PR significantly enhances the model card for `mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic` by:
- **Correcting the license** to `cc-by-4.0` as stated in the associated GitHub repository.
- **Updating the paper link** to point to the Hugging Face Papers page (`https://huggingface.co/papers/2507.11764`).
- **Adding a direct link** to the official GitHub repository (`https://github.com/MatteoFasulo/clef2025-checkthat`).
- **Enriching the metadata** with specific `languages` (Arabic, German, English, Italian, Bulgarian, Greek, Romanian, Polish, Ukrainian) and additional `tags` such as `deberta`, `multilingual`, and `subjectivity-detection` for better discoverability.
- **Populating the "Model description"**, "**Intended uses & limitations**," and "**Training and evaluation data**" sections with comprehensive details extracted from the paper abstract and GitHub README.
- **Adding a "How to use" section** with a clear Python code example for inference.
- **Including a "Citation" section** with a BibTeX entry for the paper.
These changes make the model card much more informative and user-friendly, providing essential context and practical guidance for anyone using the model.
|
@@ -1,25 +1,38 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
library_name: transformers
|
| 3 |
-
|
| 4 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
metrics:
|
| 6 |
- accuracy
|
| 7 |
- f1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
model-index:
|
| 9 |
- name: mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic
|
| 10 |
results: []
|
| 11 |
-
license: mit
|
| 12 |
-
base_model:
|
| 13 |
-
- microsoft/mdeberta-v3-base
|
| 14 |
-
pipeline_tag: text-classification
|
| 15 |
---
|
| 16 |
|
| 17 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 18 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 19 |
-
|
| 20 |
# mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic
|
| 21 |
|
| 22 |
-
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base)
|
|
|
|
|
|
|
|
|
|
| 23 |
It achieves the following results on the evaluation set:
|
| 24 |
- Loss: 0.8900
|
| 25 |
- Macro F1: 0.7969
|
|
@@ -32,15 +45,41 @@ It achieves the following results on the evaluation set:
|
|
| 32 |
|
| 33 |
## Model description
|
| 34 |
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
## Intended uses & limitations
|
| 38 |
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
## Training and evaluation data
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
## Training procedure
|
| 46 |
|
|
@@ -72,4 +111,42 @@ The following hyperparameters were used during training:
|
|
| 72 |
- Transformers 4.47.0
|
| 73 |
- Pytorch 2.5.1+cu121
|
| 74 |
- Datasets 3.3.1
|
| 75 |
-
- Tokenizers 0.21.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- microsoft/mdeberta-v3-base
|
| 4 |
library_name: transformers
|
| 5 |
+
license: cc-by-4.0
|
| 6 |
+
languages:
|
| 7 |
+
- ar
|
| 8 |
+
- de
|
| 9 |
+
- en
|
| 10 |
+
- it
|
| 11 |
+
- bg
|
| 12 |
+
- el
|
| 13 |
+
- pl
|
| 14 |
+
- ro
|
| 15 |
+
- uk
|
| 16 |
metrics:
|
| 17 |
- accuracy
|
| 18 |
- f1
|
| 19 |
+
pipeline_tag: text-classification
|
| 20 |
+
tags:
|
| 21 |
+
- generated_from_trainer
|
| 22 |
+
- deberta
|
| 23 |
+
- multilingual
|
| 24 |
+
- subjectivity-detection
|
| 25 |
model-index:
|
| 26 |
- name: mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic
|
| 27 |
results: []
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
---
|
| 29 |
|
|
|
|
|
|
|
|
|
|
| 30 |
# mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic
|
| 31 |
|
| 32 |
+
This model is a fine-tuned version of [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) for **Subjectivity Detection in News Articles**, as presented in the paper [AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles](https://huggingface.co/papers/2507.11764). It was developed by AI Wizards for their participation in the CLEF 2025 CheckThat! Lab Task 1.
|
| 33 |
+
|
| 34 |
+
**Code:** [https://github.com/MatteoFasulo/clef2025-checkthat](https://github.com/MatteoFasulo/clef2025-checkthat)
|
| 35 |
+
|
| 36 |
It achieves the following results on the evaluation set:
|
| 37 |
- Loss: 0.8900
|
| 38 |
- Macro F1: 0.7969
|
|
|
|
| 45 |
|
| 46 |
## Model description
|
| 47 |
|
| 48 |
+
This model is designed to classify sentences as **subjective** (opinion-laden) or **objective** in news articles. This task is a key component in combating misinformation, improving fact-checking pipelines, and supporting journalists.
|
| 49 |
+
|
| 50 |
+
The primary strategy of this model, based on `mDeBERTaV3-base`, involves enhancing transformer-based embeddings by integrating sentiment scores derived from an auxiliary model. This sentiment-augmented architecture significantly boosts performance, especially the subjective F1 score. The model also employs decision threshold calibration, optimized on the development set, to effectively address class imbalance prevalent across the datasets.
|
| 51 |
+
|
| 52 |
+
**Key Contributions from the paper:**
|
| 53 |
+
1. **Sentiment-Augmented Fine-Tuning**: Enriches embedding-based models by integrating sentiment scores from an auxiliary model, notably improving subjective sentence detection.
|
| 54 |
+
2. **Diverse Model Coverage**: Benchmarked across multilingual BERT variants (like mDeBERTaV3-base) for all official CLEF languages.
|
| 55 |
+
3. **Threshold Calibration for Imbalance**: A simple yet effective method to tune decision thresholds on each language's development data to enhance macro-F1 performance.
|
| 56 |
|
| 57 |
## Intended uses & limitations
|
| 58 |
|
| 59 |
+
**Intended Uses:**
|
| 60 |
+
This model is intended for subjectivity detection in news articles, classifying sentences as subjective or objective. It has been evaluated across:
|
| 61 |
+
* **Monolingual settings:** Arabic, German, English, Italian, and Bulgarian.
|
| 62 |
+
* **Zero-shot transfer:** Tested on unseen languages such as Greek, Polish, Romanian, and Ukrainian.
|
| 63 |
+
* **Multilingual training.**
|
| 64 |
+
|
| 65 |
+
Its capabilities make it suitable for applications that require discerning factual statements from opinions, such as:
|
| 66 |
+
* Combating misinformation by identifying biased or opinionated content.
|
| 67 |
+
* Improving automated fact-checking pipelines.
|
| 68 |
+
* Assisting journalists in content analysis and report generation.
|
| 69 |
+
|
| 70 |
+
**Limitations:**
|
| 71 |
+
* The model's performance is optimized for news articles; its effectiveness might vary on text from other domains (e.g., social media, conversational text) with different linguistic characteristics or content styles.
|
| 72 |
+
* While decision threshold calibration is used to mitigate class imbalance, extreme imbalances in specific datasets might still affect performance.
|
| 73 |
+
* The paper notes an initial submission error where an incorrect train/dev mix was used, leading to under-calibrated thresholds and lower initial scores in the multilingual track. While corrected results are reported, this highlights sensitivity to data preparation and calibration.
|
| 74 |
|
| 75 |
## Training and evaluation data
|
| 76 |
|
| 77 |
+
The model was fine-tuned on datasets provided for the **CLEF 2025 CheckThat! Lab Task 1: Subjectivity Detection in News Articles**.
|
| 78 |
+
|
| 79 |
+
* **Training and development datasets** were provided for monolingual settings in Arabic, German, English, Italian, and Bulgarian.
|
| 80 |
+
* **Final evaluation** included additional unseen languages such as Greek, Romanian, Polish, and Ukrainian to assess the model's generalization capabilities.
|
| 81 |
+
|
| 82 |
+
The training process involved enhancing sentence representations by integrating sentiment scores from an auxiliary model. Class imbalance, which was a notable characteristic across these languages, was addressed through decision threshold calibration during training.
|
| 83 |
|
| 84 |
## Training procedure
|
| 85 |
|
|
|
|
| 111 |
- Transformers 4.47.0
|
| 112 |
- Pytorch 2.5.1+cu121
|
| 113 |
- Datasets 3.3.1
|
| 114 |
+
- Tokenizers 0.21.0
|
| 115 |
+
|
| 116 |
+
## How to use
|
| 117 |
+
|
| 118 |
+
You can use this model directly with the Hugging Face `transformers` library for text classification:
|
| 119 |
+
|
| 120 |
+
```python
|
| 121 |
+
from transformers import pipeline
|
| 122 |
+
|
| 123 |
+
model_name = "MatteoFasulo/mdeberta-v3-base-subjectivity-sentiment-multilingual-no-arabic"
|
| 124 |
+
classifier = pipeline("text-classification", model=model_name)
|
| 125 |
+
|
| 126 |
+
# Example usage
|
| 127 |
+
text_subjective = "This is a truly amazing product and I highly recommend it!"
|
| 128 |
+
result_subjective = classifier(text_subjective)
|
| 129 |
+
print(f"'{text_subjective}' -> {result_subjective}")
|
| 130 |
+
# Expected output: [{'label': 'SUBJ', 'score': 0.99...}]
|
| 131 |
+
|
| 132 |
+
text_objective = "The capital of France is Paris."
|
| 133 |
+
result_objective = classifier(text_objective)
|
| 134 |
+
print(f"'{text_objective}' -> {result_objective}")
|
| 135 |
+
# Expected output: [{'label': 'OBJ', 'score': 0.98...}]
|
| 136 |
+
```
|
| 137 |
+
|
| 138 |
+
For more detailed usage, including training and evaluation scripts, please refer to the [GitHub repository](https://github.com/MatteoFasulo/clef2025-checkthat).
|
| 139 |
+
|
| 140 |
+
## Citation
|
| 141 |
+
|
| 142 |
+
If you find this model or the associated research useful, please consider citing the original paper:
|
| 143 |
+
|
| 144 |
+
```bibtex
|
| 145 |
+
@article{aiwizards2025checkthat,
|
| 146 |
+
title={AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles},
|
| 147 |
+
author={AI Wizards team}, # Authors not fully listed in provided context, please refer to the full paper.
|
| 148 |
+
journal={arXiv preprint arXiv:2507.11764},
|
| 149 |
+
year={2025},
|
| 150 |
+
url={https://arxiv.org/abs/2507.11764}
|
| 151 |
+
}
|
| 152 |
+
```
|