File size: 3,253 Bytes
b926950 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | # Third-party notices: toxic
`toxic` is built for commercial use. It derives from the components below,
each used under a license that permits commercial use and derivative works.
Their license terms apply to those components; the attributions here satisfy
them. Evaluation suites are never trained on.
## Base encoder
### XLM-RoBERTa base (FacebookAI)
- [`FacebookAI/xlm-roberta-base`](https://huggingface.co/FacebookAI/xlm-roberta-base): **MIT**.
- Used as the encoder: vocabulary trimmed from 250k to 60,728 pieces,
fine-tuned for multi-label hate-speech / abuse classification.
## Training datasets
- [`google/civil_comments`](https://huggingface.co/datasets/google/civil_comments)
(Civil Comments / Jigsaw Unintended Bias): **CC0-1.0** (public domain).
Courtesy citation: Borkan, Dixon, Sorensen, Thain, Vasserman, "Nuanced
Metrics for Measuring Unintended Bias with Real Data for Text
Classification", WWW 2019.
- [`ucberkeley-dlab/measuring-hate-speech`](https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech)
(Measuring Hate Speech, MHS): **CC-BY-4.0**. Attribution: D-Lab, UC
Berkeley. Citation: Kennedy, Bacon, Sahn, von Vacano, "Constructing
interval variables via faceted Rasch measurement and multitask deep
learning: a hate speech application", 2020.
- [GAHD (German Adversarial Hate speech Dataset)](https://github.com/jagol/gahd):
**CC-BY-4.0**. Citation: Goldzycher, Röttger, Schneider, "Improving
Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a
German Hate Speech Dataset", NAACL 2024.
## Datasets used during development (earlier training iterations)
- [GermEval 2018](https://github.com/uds-lsv/GermEval-2018-Data): used under
the maintainers' citation terms. Required citation: Wiegand, Siegel,
Ruppenhofer, "Overview of the GermEval 2018 Shared Task on the
Identification of Offensive Language", Proceedings of GermEval 2018,
Vienna.
- [MLMA hate speech](https://huggingface.co/datasets/nedjmaou/MLMA_hate_speech):
**MIT**. Citation: Ousidhoum, Lin, Zhang, Song, Yeung, "Multilingual and
Multi-Aspect Hate Speech Analysis", EMNLP 2019.
## Evaluation suites (eval only, never trained on)
- [HateCheck](https://github.com/paul-rottger/hatecheck-data): **CC-BY-4.0**.
Citation: Röttger, Vidgen, Nguyen, Waseem, Margetts, Pierrehumbert,
"HateCheck: Functional Tests for Hate Speech Detection Models", ACL 2021.
- [Multilingual HateCheck](https://huggingface.co/Paul/hatecheck): **CC-BY-4.0**.
Citation: Röttger, Seelawi, Nozza, Talat, Vidgen, "Multilingual HateCheck:
Functional Tests for Multilingual Hate Speech Detection Models", WOAH 2022.
## Generator models (used to create synthetic training data)
- Gemma 4 family ([`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it),
[`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it)):
**Apache-2.0**. Outputs used as synthetic multilingual training text.
- xAI `grok-4.3` (API): outputs used as synthetic training text for the
severity-tail, adversarial and clean-side minimal-pair slices, under xAI's
commercial API terms.
No non-commercial or unlicensed datasets are used. None of the above datasets
are redistributed in this repository.
|