| --- |
| language: |
| - en |
| license: cc-by-sa-4.0 |
| library_name: span-marker |
| tags: |
| - span-marker |
| - token-classification |
| - ner |
| - named-entity-recognition |
| - generated_from_span_marker_trainer |
| datasets: |
| - tomaarsen/ner-orgs |
| metrics: |
| - precision |
| - recall |
| - f1 |
| widget: |
| - text: De Napoli played for FC Luzern in the second half of the 2005–06 Swiss Super |
| League campaign, scoring five times in fifteen games and helping Luzern to promotion |
| from the Swiss Challenge League. |
| - text: The issue continued to simmer while full-communion agreements with the Presbyterian |
| Church USA, Reformed Church in America, United Church of Christ, and Episcopal |
| Church (United States) were debated and adopted in 1997 and 1999. |
| - text: Rune Gerhardsen (born 13 June 1946) is a Norwegian politician, representing |
| the Norwegian Labour Party and a former sports leader at Norwegian Skating Association |
| representing from Aktiv SK. |
| - text: Konstantin Vladimirovich Pushkaryov (; born February 12, 1985) is a Kazakhstani |
| professional ice hockey winger who is currently playing with HK Kurbads of the |
| Latvian Hockey League (LAT). |
| - text: SCL claims that its methodology has been approved or endorsed by agencies |
| of the Government of the United Kingdom and the Federal government of the United |
| States, among others. |
| pipeline_tag: token-classification |
| base_model: microsoft/xtremedistil-l12-h384-uncased |
| model-index: |
| - name: SpanMarker with microsoft/xtremedistil-l12-h384-uncased on FewNERD, CoNLL2003, |
| and OntoNotes v5 |
| results: |
| - task: |
| type: token-classification |
| name: Named Entity Recognition |
| dataset: |
| name: FewNERD, CoNLL2003, and OntoNotes v5 |
| type: tomaarsen/ner-orgs |
| split: test |
| metrics: |
| - type: f1 |
| value: 0.7558602090122487 |
| name: F1 |
| - type: precision |
| value: 0.7620428694430598 |
| name: Precision |
| - type: recall |
| value: 0.749777064383806 |
| name: Recall |
| --- |
| |
| # SpanMarker with microsoft/xtremedistil-l12-h384-uncased on FewNERD, CoNLL2003, and OntoNotes v5 |
|
|
| This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [FewNERD, CoNLL2003, and OntoNotes v5](https://huggingface.co/datasets/tomaarsen/ner-orgs) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [microsoft/xtremedistil-l12-h384-uncased](https://huggingface.co/microsoft/xtremedistil-l12-h384-uncased) as the underlying encoder. |
|
|
| ## Model Details |
|
|
| ### Model Description |
| - **Model Type:** SpanMarker |
| - **Encoder:** [microsoft/xtremedistil-l12-h384-uncased](https://huggingface.co/microsoft/xtremedistil-l12-h384-uncased) |
| - **Maximum Sequence Length:** 256 tokens |
| - **Maximum Entity Length:** 8 words |
| - **Training Dataset:** [FewNERD, CoNLL2003, and OntoNotes v5](https://huggingface.co/datasets/tomaarsen/ner-orgs) |
| - **Language:** en |
| - **License:** cc-by-sa-4.0 |
|
|
| ### Model Sources |
|
|
| - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) |
| - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) |
|
|
| ### Model Labels |
| | Label | Examples | |
| |:------|:---------------------------------------------| |
| | ORG | "Texas Chicken", "IAEA", "Church 's Chicken" | |
|
|
| ## Evaluation |
|
|
| ### Metrics |
| | Label | Precision | Recall | F1 | |
| |:--------|:----------|:-------|:-------| |
| | **all** | 0.7620 | 0.7498 | 0.7559 | |
| | ORG | 0.7620 | 0.7498 | 0.7559 | |
|
|
| ## Uses |
|
|
| ### Direct Use for Inference |
|
|
| ```python |
| from span_marker import SpanMarkerModel |
| |
| # Download from the 🤗 Hub |
| model = SpanMarkerModel.from_pretrained("nbroad/span-marker-xdistil-l12-h384-orgs-v3") |
| # Run inference |
| entities = model.predict("SCL claims that its methodology has been approved or endorsed by agencies of the Government of the United Kingdom and the Federal government of the United States, among others.") |
| ``` |
|
|
| ### Downstream Use |
| You can finetune this model on your own dataset. |
|
|
| <details><summary>Click to expand</summary> |
|
|
| ```python |
| from span_marker import SpanMarkerModel, Trainer |
| |
| # Download from the 🤗 Hub |
| model = SpanMarkerModel.from_pretrained("nbroad/span-marker-xdistil-l12-h384-orgs-v3") |
| |
| # Specify a Dataset with "tokens" and "ner_tag" columns |
| dataset = load_dataset("conll2003") # For example CoNLL2003 |
| |
| # Initialize a Trainer using the pretrained model & dataset |
| trainer = Trainer( |
| model=model, |
| train_dataset=dataset["train"], |
| eval_dataset=dataset["validation"], |
| ) |
| trainer.train() |
| trainer.save_model("nbroad/span-marker-xdistil-l12-h384-orgs-v3-finetuned") |
| ``` |
| </details> |
|
|
| <!-- |
| ### Out-of-Scope Use |
|
|
| *List how the model may foreseeably be misused and address what users ought not to do with the model.* |
| --> |
|
|
| <!-- |
| ## Bias, Risks and Limitations |
|
|
| *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
| --> |
|
|
| <!-- |
| ### Recommendations |
|
|
| *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
| --> |
|
|
| ## Training Details |
|
|
| ### Training Set Metrics |
| | Training set | Min | Median | Max | |
| |:----------------------|:----|:--------|:----| |
| | Sentence length | 1 | 23.5706 | 263 | |
| | Entities per sentence | 0 | 0.7865 | 39 | |
|
|
| ### Training Hyperparameters |
| - learning_rate: 0.0003 |
| - train_batch_size: 128 |
| - eval_batch_size: 128 |
| - seed: 42 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - lr_scheduler_warmup_ratio: 0.05 |
| - num_epochs: 3 |
| - mixed_precision_training: Native AMP |
| |
| ### Training Results |
| | Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy | |
| |:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:| |
| | 0.5720 | 600 | 0.0086 | 0.7150 | 0.7095 | 0.7122 | 0.9660 | |
| | 1.1439 | 1200 | 0.0074 | 0.7556 | 0.7253 | 0.7401 | 0.9682 | |
| | 1.7159 | 1800 | 0.0073 | 0.7482 | 0.7619 | 0.7550 | 0.9702 | |
| | 2.2879 | 2400 | 0.0072 | 0.7761 | 0.7573 | 0.7666 | 0.9713 | |
| | 2.8599 | 3000 | 0.0070 | 0.7691 | 0.7688 | 0.7689 | 0.9720 | |
| |
| ### Framework Versions |
| - Python: 3.10.12 |
| - SpanMarker: 1.5.0 |
| - Transformers: 4.35.2 |
| - PyTorch: 2.1.0a0+32f93b1 |
| - Datasets: 2.15.0 |
| - Tokenizers: 0.15.0 |
| |
| ## Citation |
| |
| ### BibTeX |
| ``` |
| @software{Aarsen_SpanMarker, |
| author = {Aarsen, Tom}, |
| license = {Apache-2.0}, |
| title = {{SpanMarker for Named Entity Recognition}}, |
| url = {https://github.com/tomaarsen/SpanMarkerNER} |
| } |
| ``` |
| |
| <!-- |
| ## Glossary |
|
|
| *Clearly define terms in order to be accessible across audiences.* |
| --> |
|
|
| <!-- |
| ## Model Card Authors |
|
|
| *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
| --> |
|
|
| <!-- |
| ## Model Card Contact |
|
|
| *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
| --> |
