---
language:
  - en
license: mit
tags:
  - deberta
  - deberta-v3
  - stance-classification
  - nli
  - zero-shot-classification
  - political-science
  - social-groups
  - parliamentary-debates
base_model: MoritzLaurer/deberta-v3-base-zeroshot-v2.0
pipeline_tag: zero-shot-classification
library_name: transformers
metrics:
  - f1
  - precision
  - recall
---

# Social Group Stance Classification (NLI)

A DeBERTa-v3-base NLI model fine-tuned to classify the author's **stance toward a social group** in a political text as **positive**, **negative**, or **neutral**.

This model is part of the [`group-appeal-detector`](https://github.com/MaximilianWeiland/group_appeal_detector) package, which also provides group mention detection and mention clustering.

## Model Details

- **Base model:** [`MoritzLaurer/deberta-v3-base-zeroshot-v2.0`](https://huggingface.co/MoritzLaurer/deberta-v3-base-zeroshot-v2.0)
- **Task:** Stance Classification (NLI-based)
- **Labels:** `positive`, `negative`, `neutral`
- **Training data:** 5,000 manually annotated sentences from UK House of Commons parliamentary debates (2010–2019), with the negative class oversampled by adding synthetic paraphrases for half of all negative social group appeals

## How It Works

For each detected group mention, three hypotheses are formulated — one per stance class. The model chooses the class with the largest entailment probability as the predicted stance.

## Performance

Cross-validated performance (95% confidence intervals based on the estimated standard error across folds in brackets):

| Metric    | Negative          | Neutral           | Positive          | Macro-Avg.        |
|-----------|-------------------|-------------------|-------------------|-------------------|
| F1        | 0.76 [0.72, 0.80] | 0.80 [0.78, 0.81] | 0.89 [0.89, 0.89] | 0.81 [0.80, 0.83] |
| Precision | 0.85 [0.77, 0.94] | 0.81 [0.79, 0.84] | 0.87 [0.86, 0.88] | 0.85 [0.82, 0.87] |
| Recall    | 0.70 [0.62, 0.77] | 0.78 [0.76, 0.80] | 0.91 [0.89, 0.92] | 0.79 [0.77, 0.82] |

## Usage

### Via `group-appeal-detector` package (recommended)

```bash
pip install group-appeal-detector
```

```python
from group_appeal_detector import GroupAppealDetector

detector = GroupAppealDetector(device="cpu")

sentence = "We must protect the rights of farmers."
target_group = "farmers"
result = detector.classify_stance(sentence, target_group)
print(result["predicted_stance"], result["stance_probs"])
```

For batch processing:

```python
pairs = [
    ("We must protect the rights of farmers.", "farmers"),
    ("We do a lot for elderly people.", "elderly people"),
]
results_df = detector.classify_stance_batch(pairs, batch_size=8, as_df=True)
```

### Direct usage with Transformers

```python
from transformers import pipeline

pipe = pipeline(
    "zero-shot-classification",
    model="maxwlnd/socialgroup_stance_classification_nli",
)

sentence = "We must protect the rights of farmers."
target_group = "farmers"

hypotheses = [
    f"The text is positive towards {target_group}.",
    f"The text is negative towards {target_group}.",
    f"The text is neutral, or contains no stance, towards {target_group}.",
]

result = pipe(sentence, candidate_labels=hypotheses)
print(result["labels"][0], result["scores"][0])
```

## Related Models

This model is one of three models in the group appeal detection pipeline:

| Model | Task |
|---|---|
| [`maxwlnd/roberta_group_mention_detector`](https://huggingface.co/maxwlnd/roberta_group_mention_detector) | Detect social group mentions |
| [`maxwlnd/socialgroup_stance_classification_nli`](https://huggingface.co/maxwlnd/socialgroup_stance_classification_nli) | Classify stance toward a group as positive, negative, or neutral (this model) |
| [`maxwlnd/cl_mention_embedding`](https://huggingface.co/maxwlnd/cl_mention_embedding) | Embed mentions for clustering into qualitative categories |

## Conceptual Background

The definition of a social group appeal used for annotating the training data is inspired by [Lena Maria Huber and Alona O. Dolinsky](https://osf.io/preprints/osf/szaqw_v1) and [Will Horne, Alona O. Dolinsky and Lena Maria Huber](https://osf.io/preprints/osf/fp2h3_v3).

A **group appeal** is an intentional act by a political actor that associates them with a social group in a supportive or critical manner. This model classifies whether the author's expressed stance toward the mentioned group is positive, negative, or neutral.

## License

MIT