amitca71 commited on
Commit
06efd55
ยท
verified ยท
1 Parent(s): e6b8b0f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +36 -51
README.md CHANGED
@@ -2,90 +2,75 @@
2
  language: ar
3
  tags:
4
  - text-classification
5
- - sentiment
6
  - levantine
 
7
  - marbert
8
- - hate-speech
9
  metrics:
10
  - accuracy
11
- - f1
12
  model-index:
13
- - name: Levantine Toxic Classifier
14
  results:
15
  - task:
16
  type: text-classification
17
- name: Text Classification
18
  dataset:
19
  name: ArSenTD-LEV
20
  type: amitca71/marabert2-levantine-sentiment-model-dataset
21
  metrics:
22
  - type: accuracy
23
  value: 79.50%
24
- - type: f1
25
- value: 0.76
26
  ---
27
 
28
- # ๐Ÿ›ก๏ธ Levantine Hate Speech Classifier (v2)
29
 
30
- This model is a fine-tuned version of **MARBERTv2**, designed to detect **Hate Speech** and **Abusive Language** in Levantine Arabic tweets.
31
 
32
- It is specifically trained to handle **Sarcasm** and **Indirect Threats** (e.g., "The grave is waiting" - ุงู„ู‚ุจุฑ ู†ุงุทุฑ) which are often missed by standard models.
33
 
34
  ## ๐Ÿ“Š Performance
35
  | Metric | Score | Description |
36
  | :--- | :--- | :--- |
37
  | **Accuracy** | **79.50%** | Overall correctness on the test set. |
38
- | **F1 (Toxic)** | **0.76** | Ability to catch toxic tweets without false alarms. |
39
 
40
  ## ๐Ÿ“– Labels
41
  | ID | Label | Meaning |
42
  | :--- | :--- | :--- |
43
- | **0** | **Normal** | Safe, standard conversation (including non-hateful insults). |
44
- | **1** | **Toxic** | Hate speech, violence, or identity-based attacks. |
 
45
 
46
  ## ๐Ÿš€ How to Use (Python)
47
 
48
- This model includes a **custom threshold** optimization. For best results, use the code below to load the threshold automatically from the config.
49
 
50
 
51
  ```python
52
- from transformers import pipeline, AutoConfig
53
-
54
- # 1. Load Model & Config
55
- model_id = "amitca71/marabert2-levantine-toxic-model"
56
- config = AutoConfig.from_pretrained(model_id)
57
-
58
- # 2. Get the Optimized Threshold (or default to 0.5)
59
- THRESHOLD = getattr(config, "custom_threshold", 0.5)
60
- print(f"โš™๏ธ Using Threshold: {THRESHOLD}")
61
-
62
- # 3. Load Pipeline
63
- classifier = pipeline("text-classification", model=model_id, top_k=None)
64
-
65
- def predict(text):
66
- # Get all scores (Normal + Toxic)
67
- output = classifier(text)
68
-
69
- # Handle list-of-lists vs list-of-dicts format
70
- scores = output[0] if isinstance(output[0], list) else output
71
-
72
- # Find the Toxic Score safely
73
- toxic_score = 0.0
74
- for item in scores:
75
- if item['label'].lower() in ['toxic', 'label_1', '1']:
76
- toxic_score = item['score']
77
- break
78
-
79
- # Apply Threshold
80
- label = "TOXIC โš ๏ธ" if toxic_score > THRESHOLD else "Normal โœ…"
81
-
82
- return {"text": text, "label": label, "score": round(toxic_score, 4)}
83
-
84
- # 4. Test
85
- print(predict("ูŠุง ุฒู„ู…ุฉ ุญู„ ุนู†ูŠ ูŠุง ุบุจูŠ")) # Likely Normal (Insult)
86
- print(predict("ุงู„ู‚ุจุฑ ู†ุงุทุฑ")) # Likely Toxic (Threat)
87
  ```
88
 
89
  ## โš ๏ธ Limitations
90
- * **Dialect:** Optimized for Levantine (Jordan, Lebanon, Palestine, Syria). May underperform on Egyptian or Gulf dialects.
91
- * **Context:** Very short tweets (<3 words) may lack sufficient context for accurate sarcasm detection.
 
 
2
  language: ar
3
  tags:
4
  - text-classification
5
+ - sentiment-analysis
6
  - levantine
7
+ - arabic
8
  - marbert
9
+ - ordinal-classification
10
  metrics:
11
  - accuracy
12
+ - f1_macro
13
  model-index:
14
+ - name: Levantine Sentiment Classifier (Ordinal)
15
  results:
16
  - task:
17
  type: text-classification
18
+ name: Sentiment Analysis
19
  dataset:
20
  name: ArSenTD-LEV
21
  type: amitca71/marabert2-levantine-sentiment-model-dataset
22
  metrics:
23
  - type: accuracy
24
  value: 79.50%
25
+ - type: f1_macro
26
+ value: 0.7635
27
  ---
28
 
29
+ # ๐ŸŽญ Levantine Arabic Sentiment Classifier (Ordinal MARBERTv2)
30
 
31
+ This model is a fine-tuned version of **MARBERTv2**, designed to predict the sentiment of Levantine Arabic tweets (Jordanian, Lebanese, Palestinian, Syrian).
32
 
33
+ **Technical Highlight:** This model was trained using an **Ordinal Loss Function** (Mean Squared Error combined with Cross-Entropy). This makes the model "distance-aware," meaning it heavily penalizes extreme mistakes (like confusing a highly positive tweet for a highly negative one). This makes its predictions far more reliable in edge cases!
34
 
35
  ## ๐Ÿ“Š Performance
36
  | Metric | Score | Description |
37
  | :--- | :--- | :--- |
38
  | **Accuracy** | **79.50%** | Overall correctness on the test set. |
39
+ | **F1 (Macro)** | **0.7635** | The balanced F1 score across all 3 classes. |
40
 
41
  ## ๐Ÿ“– Labels
42
  | ID | Label | Meaning |
43
  | :--- | :--- | :--- |
44
+ | **0** | **Negative ๐Ÿ˜ ** | Anger, complaints, sadness, or frustration. |
45
+ | **1** | **Neutral ๐Ÿ˜** | Objective facts, mixed emotions, or ambiguous statements. |
46
+ | **2** | **Positive ๐Ÿ˜ƒ** | Joy, praise, excitement, or satisfaction. |
47
 
48
  ## ๐Ÿš€ How to Use (Python)
49
 
50
+ Because this is a standard 3-class model, you can easily load it using Hugging Face's built-in `pipeline`.
51
 
52
 
53
  ```python
54
+ from transformers import pipeline
55
+
56
+ # 1. Load Pipeline
57
+ model_id = "amitca71/marabert2-levantine-sentiment"
58
+ classifier = pipeline("text-classification", model=model_id, top_k=1)
59
+
60
+ def predict_sentiment(text):
61
+ # Get the top prediction
62
+ result = classifier(text)[0]
63
+
64
+ # Format the output cleanly
65
+ return {"text": text, "label": result['label'], "confidence": round(result['score'], 4)}
66
+
67
+ # 2. Test Examples
68
+ print(predict_sentiment("ุงู„ุฌูˆ ุงู„ูŠูˆู… ุจูŠุนู‚ุฏ! ุทุงู„ุนูŠู† ู…ุดูˆุงุฑ")) # Should be Positive
69
+ print(predict_sentiment("ูˆุงู„ู„ู‡ ุทู‚ุช ุฑูˆุญูŠ ู…ู† ู‡ุงู„ุฒุญู…ุฉุŒ ุดูŠ ุจูŠู‚ุฑู")) # Should be Negative
70
+ print(predict_sentiment("ูˆุตู„ุช ุนุงู„ุจูŠุช ู…ู† ุดูˆูŠ.")) # Should be Neutral
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  ```
72
 
73
  ## โš ๏ธ Limitations
74
+ * **Dialect Focus:** Optimized heavily for Levantine Twitter. It may underperform or misunderstand idioms in Egyptian, Gulf, or Maghrebi dialects.
75
+ * **The "Neutral" Bottleneck:** Like most sentiment models, detecting true "Neutral" text is the most challenging, as human annotators often mix objective facts with subtle sarcasm in this category.
76
+ * **Arabizi:** While MARBERTv2 has some exposure to Arabizi (Arabic written in English/Latin letters), this model performs best on native Arabic script.