toxic / THIRD_PARTY_NOTICES.md
pveugen's picture
toxic v0.1.0
b926950
|
Raw
History Blame Contribute Delete
3.25 kB

Third-party notices: toxic

toxic is built for commercial use. It derives from the components below, each used under a license that permits commercial use and derivative works. Their license terms apply to those components; the attributions here satisfy them. Evaluation suites are never trained on.

Base encoder

XLM-RoBERTa base (FacebookAI)

  • FacebookAI/xlm-roberta-base: MIT.
  • Used as the encoder: vocabulary trimmed from 250k to 60,728 pieces, fine-tuned for multi-label hate-speech / abuse classification.

Training datasets

  • google/civil_comments (Civil Comments / Jigsaw Unintended Bias): CC0-1.0 (public domain). Courtesy citation: Borkan, Dixon, Sorensen, Thain, Vasserman, "Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification", WWW 2019.
  • ucberkeley-dlab/measuring-hate-speech (Measuring Hate Speech, MHS): CC-BY-4.0. Attribution: D-Lab, UC Berkeley. Citation: Kennedy, Bacon, Sahn, von Vacano, "Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application", 2020.
  • GAHD (German Adversarial Hate speech Dataset): CC-BY-4.0. Citation: Goldzycher, Röttger, Schneider, "Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset", NAACL 2024.

Datasets used during development (earlier training iterations)

  • GermEval 2018: used under the maintainers' citation terms. Required citation: Wiegand, Siegel, Ruppenhofer, "Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language", Proceedings of GermEval 2018, Vienna.
  • MLMA hate speech: MIT. Citation: Ousidhoum, Lin, Zhang, Song, Yeung, "Multilingual and Multi-Aspect Hate Speech Analysis", EMNLP 2019.

Evaluation suites (eval only, never trained on)

  • HateCheck: CC-BY-4.0. Citation: Röttger, Vidgen, Nguyen, Waseem, Margetts, Pierrehumbert, "HateCheck: Functional Tests for Hate Speech Detection Models", ACL 2021.
  • Multilingual HateCheck: CC-BY-4.0. Citation: Röttger, Seelawi, Nozza, Talat, Vidgen, "Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models", WOAH 2022.

Generator models (used to create synthetic training data)

  • Gemma 4 family (google/gemma-4-E4B-it, google/gemma-4-12B-it): Apache-2.0. Outputs used as synthetic multilingual training text.
  • xAI grok-4.3 (API): outputs used as synthetic training text for the severity-tail, adversarial and clean-side minimal-pair slices, under xAI's commercial API terms.

No non-commercial or unlicensed datasets are used. None of the above datasets are redistributed in this repository.