--- language: - en - de - fr - it - es license: mit tags: - token-classification - ner - multilingual - gliner - sauerkrautlm - pii library_name: gliner pipeline_tag: token-classification --- # SauerkrautLM-GLiNER SauerkrautLM-GLiNER is a multilingual GLiNER-style model for **zero-shot named entity recognition (NER)** based on the `jhu-clsp/mmBERT-base` backbone (a ModernBERT-style multilingual encoder). The model is trained jointly on **English, German, Italian, French, and Spanish** over roughly **21k distinct entity types**, and is designed for **prompted / label-list NER** in real-world multilingual scenarios (documents, logs, PII detection, etc.). As a baseline, we compare against **`urchade/gliner_multi-v2.1`**, which is built on **`microsoft/mdeberta-v3-base`**, a multilingual DeBERTa v3 model that extends BERT/RoBERTa with **disentangled attention** and an **enhanced mask decoder**, architectural features that improve token-level representations and benefit sequence-labeling tasks such as NER. # Demo > 🔍 **Demo:** You can try the model directly in the browser via our Hugging Face Space: > **https://huggingface.co/spaces/VAGOsolutions/SauerkrautLM-GLiNER-Demo** --- ## Model details - **Model name:** SauerkrautLM-GLiNER - **Backbone:** `jhu-clsp/mmBERT-base` (ModernBERT-style multilingual encoder) - **Architecture:** GLiNER-style prompted NER (span classification conditioned on textual labels) - **Languages (training):** English, German, French, Italian, Spanish - **Entity types:** ~21k distinct entity labels - **Objective:** Multilingual, **zero-shot** and **few-shot** NER with arbitrary label lists - **License:** MIT The model is trained as a **general-purpose multilingual NER model**: unlike a PII-only model, it is exposed to many domains (news, encyclopedic text, technical and general content) and a very large label space, making it suitable for both **broad-domain extraction** and **specialized taxonomies**. ## Evaluation & Benchmarks ### Datasets The model is evaluated on the **VAGOsolutions/gliner-benchmark-multilingual** dataset ([🤗 Dataset Link](https://huggingface.co/datasets/VAGOsolutions/gliner-benchmark-multilingual)), which extends the original **DFKI-SLT/cross_ner** dataset (English-only) with **German, French, Italian, and Spanish**. **Important:** The non-English data is **not simply translated** from the English original. Instead, new language-specific data was carefully curated to ensure high-quality, authentic entity extraction across all languages. ### Results We compare **SauerkrautLM-GLiNER** against two baselines: - **urchade/gliner_multi-v2.1** – General multilingual NER model - **urchade/gliner_multi_pii-v1** – Specialized PII detection model #### CrossNER + Multilingual Domain Benchmarks (Threshold: 0.8 for SauerkrautLM-GLiNER, 0.65 for gliner_multi-v2.1) | Dataset | Samples | Entities | **SauerkrautLM-GLiNER**
P / R / F1 | **gliner_multi-v2.1**
P / R / F1 | |---------|---------|----------|----------------------------------------|--------------------------------------| | **CrossNER (English)** | | | | | | crossner_ai_test.jsonl | 431 | 14 | 58.03 / 48.92 / **53.09** | 64.94 / 40.24 / 49.69 | | crossner_literature_test.jsonl | 416 | 12 | 55.57 / 47.79 / **51.39** | 73.11 / 51.59 / 60.49 | | crossner_music_test.jsonl | 465 | 13 | 67.43 / 54.86 / **60.50** | 76.80 / 66.40 / 71.22 | | crossner_politics_test.jsonl | 651 | 9 | 57.85 / 54.36 / **56.05** | 72.85 / 64.46 / 68.40 | | crossner_science_test.jsonl | 543 | 17 | 57.83 / 47.01 / **51.86** | 67.05 / 54.10 / 59.88 | | **AI Domain (Multilingual)** | | | | | | ai_french.jsonl | 2311 | 20 | 53.76 / 66.13 / **59.30** | 56.68 / 22.17 / 31.88 | | ai_german.jsonl | 2348 | 19 | 53.37 / 67.99 / **59.80** | 57.41 / 23.23 / 33.08 | | ai_italian.jsonl | 2188 | 25 | 47.33 / 54.00 / **50.45** | 49.64 / 15.62 / 23.76 | | ai_spanish.jsonl | 2253 | 19 | 54.23 / 63.96 / **58.69** | 50.09 / 21.63 / 30.21 | | **Literature Domain (Multilingual)** | | | | | | literature_french.jsonl | 541 | 17 | 45.64 / 59.36 / **51.60** | 36.75 / 18.16 / 24.31 | | literature_german.jsonl | 505 | 20 | 34.59 / 41.94 / **37.92** | 30.13 / 10.04 / 15.07 | | literature_italian.jsonl | 634 | 28 | 47.22 / 59.13 / **52.51** | 23.89 / 5.39 / 8.79 | | literature_spanish.jsonl | 495 | 17 | 47.21 / 58.73 / **52.34** | 37.14 / 16.81 / 23.14 | | **Music Domain (Multilingual)** | | | | | | music_french.jsonl | 576 | 18 | 55.59 / 67.62 / **61.02** | 45.58 / 26.11 / 33.20 | | music_german.jsonl | 457 | 18 | 53.68 / 65.01 / **58.81** | 54.95 / 31.94 / 40.40 | | music_italian.jsonl | 555 | 23 | 71.75 / 73.07 / **72.40** | 45.48 / 16.92 / 24.66 | | music_spanish.jsonl | 540 | 18 | 59.54 / 68.54 / **63.72** | 49.07 / 27.54 / 35.28 | | **Politics Domain (Multilingual)** | | | | | | politics_french.jsonl | 571 | 11 | 44.44 / 64.59 / **52.65** | 40.94 / 27.31 / 32.76 | | politics_german.jsonl | 550 | 14 | 31.96 / 46.44 / **37.86** | 25.16 / 13.88 / 17.90 | | politics_italian.jsonl | 574 | 17 | 49.64 / 68.87 / **57.70** | 41.52 / 21.04 / 27.93 | | politics_spanish.jsonl | 559 | 13 | 50.86 / 69.71 / **58.81** | 46.14 / 29.71 / 36.15 | | **Science Domain (Multilingual)** | | | | | | science_french.jsonl | 563 | 25 | 59.24 / 61.42 / **60.31** | 55.55 / 13.89 / 22.23 | | science_german.jsonl | 555 | 29 | 41.79 / 49.61 / **45.37** | 34.55 / 8.99 / 14.27 | | science_italian.jsonl | 523 | 36 | 64.55 / 63.35 / **63.95** | 43.21 / 8.18 / 13.75 | | science_spanish.jsonl | 555 | 32 | 54.95 / 55.95 / **55.45** | 23.89 / 5.88 / 9.43 | | **AVERAGE** | **20,359** | – | **52.72 / 59.13 / 55.34** | **48.10 / 25.65 / 32.32** | **Key Takeaway:** SauerkrautLM-GLiNER achieves **+23.02 F1 points** over gliner_multi-v2.1 on average, with particularly strong improvements in recall across all non-English languages. --- #### PII Detection Benchmark (Threshold: 0.75 for SauerkrautLM-GLiNER, 0.6 for baselines) | Dataset | Samples | Entities | **SauerkrautLM-GLiNER**
P / R / F1 | **gliner_multi-v2.1**
P / R / F1 | **gliner_multi_pii-v1**
P / R / F1 | |---------|---------|----------|----------------------------------------|--------------------------------------|----------------------------------------| | pii_english.jsonl | 500 | 20 | 55.83 / 40.86 / **47.19** | 32.12 / 16.01 / 21.37 | 63.33 / 26.74 / 37.60 | | pii_french.jsonl | 500 | 20 | 56.62 / 36.44 / **44.34** | 36.08 / 18.09 / 24.09 | 71.57 / 37.14 / 48.90 | | pii_german.jsonl | 500 | 20 | 54.59 / 36.24 / **43.56** | 34.61 / 16.85 / 22.67 | 69.48 / 31.41 / 43.27 | | pii_italian.jsonl | 500 | 20 | 58.14 / 38.01 / **45.97** | 39.80 / 19.93 / 26.56 | 71.40 / 34.38 / 46.41 | | pii_spanish.jsonl | 500 | 20 | 55.05 / 36.13 / **43.62** | 39.86 / 20.24 / 26.85 | 66.84 / 33.25 / 44.41 | | **AVERAGE** | **2,500** | – | **56.05 / 37.54 / 44.94** | **36.49 / 18.22 / 24.31** | **68.52 / 32.58 / 44.12** | **Key Takeaway:** SauerkrautLM-GLiNER performs competitively on PII detection despite being a general-purpose NER model (not PII-specialized), achieving **+20.63 F1** over gliner_multi-v2.1 and matching the specialized gliner_multi_pii-v1 model. --- ## Usage ### Installation ```bash pip install gliner ``` ### Basic Example ```python from gliner import GLiNER # Load the model model = GLiNER.from_pretrained("VAGOsolutions/SauerkrautLM-GLiNER") # Example text text = """ Cristiano Ronaldo dos Santos Aveiro (Portuguese pronunciation: [kɾiʃˈtjɐnu ʁɔˈnaldu]; born 5 February 1985) is a Portuguese professional footballer who plays as a forward for and captains both Saudi Pro League club Al Nassr and the Portugal national team. Widely regarded as one of the greatest players of all time, Ronaldo has won five Ballon d'Or awards,[note 3] a record three UEFA Men's Player of the Year Awards, and four European Golden Shoes, the most by a European player. He has won 33 trophies in his career, including seven league titles, five UEFA Champions Leagues, the UEFA European Championship and the UEFA Nations League. Ronaldo holds the records for most appearances (183), goals (140) and assists (42) in the Champions League, goals in the European Championship (14), international goals (128) and international appearances (205). He is one of the few players to have made over 1,200 professional career appearances, the most by an outfield player, and has scored over 850 official senior career goals for club and country, making him the top goalscorer of all time. """ # Define labels (entity types you want to extract) labels = ["person", "award", "date", "competitions", "teams"] # Predict entities entities = model.predict_entities(text, labels) # Print results for entity in entities: print(entity["text"], "=>", entity["label"]) ``` **Expected Output:** ``` Cristiano Ronaldo dos Santos Aveiro => person 5 February 1985 => date Al Nassr => teams Portugal national team => teams Ballon d'Or => award UEFA Men's Player of the Year Awards => award European Golden Shoes => award UEFA Champions Leagues => competitions UEFA European Championship => competitions UEFA Nations League => competitions Champions League => competitions European Championship => competitions ``` ### Multilingual Example The model works seamlessly across multiple languages: ```python # German text text_de = "Angela Merkel war von 2005 bis 2021 Bundeskanzlerin der Bundesrepublik Deutschland." labels_de = ["person", "position", "location", "date"] entities_de = model.predict_entities(text_de, labels_de) for entity in entities_de: print(entity["text"], "=>", entity["label"]) # French text text_fr = "Emmanuel Macron est président de la République française depuis mai 2017." labels_fr = ["person", "position", "location", "date"] entities_fr = model.predict_entities(text_fr, labels_fr) for entity in entities_fr: print(entity["text"], "=>", entity["label"]) ``` ### Custom Labels for Domain-Specific NER Simply adjust the label list to match your domain: ```python # PII Detection text_pii = "Contact John Smith at john.smith@email.com or call +1-555-0123." labels_pii = ["person", "email", "phone"] # Scientific Text text_science = "The experiment used CRISPR-Cas9 to edit the BRCA1 gene in human cells." labels_science = ["technique", "gene", "organism"] # Business Documents text_business = "Apple Inc. acquired startup.ai for $2.5 billion in Q3 2024." labels_business = ["company", "money", "date"] ``` --- ## Intended use & limitations ### Intended use - Zero-shot or few-shot NER in: - Multilingual documents (EN/DE/FR/IT/ES) - Domain-specific content (AI, literature, music, politics, science) - PII / sensitive-data detection (emails, IDs, names, locations, etc.) - Rapid prototyping of new NER schemas by simply changing the **label list**: - e.g. `["person", "organization", "product"]` - or more domain-specific labels like `["award", "competition", "team", "date"]` ### Not intended for - Safety-critical scenarios without human oversight (e.g. automated compliance decisions or legal / financial judgments). --- ## Citation If you use SauerkrautLM-GLiNER in your research or applications, please cite: ```bibtex @misc{SauerkrautLM-GLiNER, title={SauerkrautLM-GLiNER}, author={Michele Montebovi}, organization={VAGO Solutions}, url={https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNER}, year={2025} } ``` ### Acknowledgments This model builds upon: - **mmBERT** ([jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base)) – ModernBERT-style multilingual encoder - **GLiNER** architecture ([urchade/gliner_multi-v2.1](https://huggingface.co/urchade/gliner_multi-v2.1)) – Zero-shot NER framework