Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -16,10 +16,10 @@ pipeline_tag: token-classification
|
|
| 16 |
|
| 17 |
## Model Description
|
| 18 |
|
| 19 |
-
This is a Telugu-language model based on [
|
| 20 |
|
| 21 |
-
- sentence segmentation (via [
|
| 22 |
-
- tokenisation (via [
|
| 23 |
- part-of-speech tagging
|
| 24 |
- morphological analysis
|
| 25 |
- lemmatisation
|
|
@@ -32,22 +32,22 @@ The Telugu model uses ``FacebookAI/xlm-roberta-base`` as its base encoder and is
|
|
| 32 |
Evaluation was performed on the UD_Telugu-MTG test split using the standard [CoNLL 2018 eval script](https://universaldependencies.org/conll18/conll18_ud_eval.py).
|
| 33 |
|
| 34 |
Two evaluation rows are reported:
|
| 35 |
-
- **Full-text (F1)**: raw text is segmented by [
|
| 36 |
- **Aligned accuracy**: accuracy on correctly segmented (aligned) tokens — measures parsing quality on tokens that were correctly identified by the segmenter.
|
| 37 |
|
| 38 |
### Morphosyntactic Tagging
|
| 39 |
|
| 40 |
| Metric | Tokens | Sentences | Words | UPOS | XPOS | UFeats | AllTags | Lemmas |
|
| 41 |
| ------ | ------ | --------- | ----- | ---- | ---- | ------ | ------- | ------ |
|
| 42 |
-
| Full-text (F1) | 99.79 |
|
| 43 |
-
| Aligned accuracy | 0.00 | 0.00 | 0.00 | 95.
|
| 44 |
|
| 45 |
### Dependency Parsing
|
| 46 |
|
| 47 |
| Metric | UAS | LAS | CLAS | MLAS | BLEX |
|
| 48 |
| ------ | --- | --- | ---- | ---- | ---- |
|
| 49 |
-
| Full-text (F1) |
|
| 50 |
-
| Aligned accuracy |
|
| 51 |
|
| 52 |
|
| 53 |
## Usage
|
|
@@ -58,16 +58,14 @@ Install the library from PyPI (assuming you have a virtual environment created):
|
|
| 58 |
pip install combo-nlp
|
| 59 |
```
|
| 60 |
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
pip install --index-url https://pypi.clarin-pl.eu/ lambo
|
| 65 |
-
```
|
| 66 |
|
| 67 |
```python
|
| 68 |
from combo import COMBO
|
| 69 |
|
| 70 |
-
# Load a pre-trained model with corresponding
|
| 71 |
nlp = COMBO("Telugu")
|
| 72 |
|
| 73 |
# Parse raw text (handles sentence splitting + tokenization)
|
|
@@ -79,10 +77,10 @@ for sentence in result:
|
|
| 79 |
print(f"{token.form:<15} {token.lemma:<15} {token.upos:<8} head={token.head} {token.deprel}")
|
| 80 |
```
|
| 81 |
|
| 82 |
-
Refer to the
|
| 83 |
|
| 84 |
-
- [https://
|
| 85 |
-
- [https://
|
| 86 |
|
| 87 |
## License
|
| 88 |
|
|
@@ -112,7 +110,7 @@ Ulewicz, M., Jabłońska, M., Klimaszewski, M., Przybyła, P., Pszenny, Ł., Ryb
|
|
| 112 |
|
| 113 |
## Resources
|
| 114 |
|
| 115 |
-
-
|
| 116 |
-
-
|
| 117 |
- UD_Telugu-MTG: [https://github.com/UniversalDependencies/UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG)
|
| 118 |
|
|
|
|
| 16 |
|
| 17 |
## Model Description
|
| 18 |
|
| 19 |
+
This is a Telugu-language model based on [combo-nlp](https://pypi.org/project/combo-nlp/), an open-source natural language preprocessing system. It performs:
|
| 20 |
|
| 21 |
+
- sentence segmentation (via [combo-seg](https://pypi.org/project/combo-seg/))
|
| 22 |
+
- tokenisation (via [combo-seg](https://pypi.org/project/combo-seg/))
|
| 23 |
- part-of-speech tagging
|
| 24 |
- morphological analysis
|
| 25 |
- lemmatisation
|
|
|
|
| 32 |
Evaluation was performed on the UD_Telugu-MTG test split using the standard [CoNLL 2018 eval script](https://universaldependencies.org/conll18/conll18_ud_eval.py).
|
| 33 |
|
| 34 |
Two evaluation rows are reported:
|
| 35 |
+
- **Full-text (F1)**: raw text is segmented by [combo-seg](https://pypi.org/project/combo-seg/), then parsed and compared against gold — measures end-to-end pipeline performance including segmentation quality.
|
| 36 |
- **Aligned accuracy**: accuracy on correctly segmented (aligned) tokens — measures parsing quality on tokens that were correctly identified by the segmenter.
|
| 37 |
|
| 38 |
### Morphosyntactic Tagging
|
| 39 |
|
| 40 |
| Metric | Tokens | Sentences | Words | UPOS | XPOS | UFeats | AllTags | Lemmas |
|
| 41 |
| ------ | ------ | --------- | ----- | ---- | ---- | ------ | ------- | ------ |
|
| 42 |
+
| Full-text (F1) | 99.79 | 97.58 | 99.79 | 94.80 | 94.80 | 98.96 | 94.53 | 99.79 |
|
| 43 |
+
| Aligned accuracy | 0.00 | 0.00 | 0.00 | 95.00 | 95.00 | 99.17 | 94.72 | 100.00 |
|
| 44 |
|
| 45 |
### Dependency Parsing
|
| 46 |
|
| 47 |
| Metric | UAS | LAS | CLAS | MLAS | BLEX |
|
| 48 |
| ------ | --- | --- | ---- | ---- | ---- |
|
| 49 |
+
| Full-text (F1) | 90.64 | 83.16 | 79.43 | 76.19 | 79.43 |
|
| 50 |
+
| Aligned accuracy | 90.83 | 83.33 | 79.58 | 76.34 | 79.58 |
|
| 51 |
|
| 52 |
|
| 53 |
## Usage
|
|
|
|
| 58 |
pip install combo-nlp
|
| 59 |
```
|
| 60 |
|
| 61 |
+
The [combo-seg](https://pypi.org/project/combo-seg/) segmenter (used to split and
|
| 62 |
+
tokenise raw text) is installed automatically as a dependency of `combo-nlp`, so
|
| 63 |
+
no extra install step is needed.
|
|
|
|
|
|
|
| 64 |
|
| 65 |
```python
|
| 66 |
from combo import COMBO
|
| 67 |
|
| 68 |
+
# Load a pre-trained model with the corresponding combo-seg segmenter
|
| 69 |
nlp = COMBO("Telugu")
|
| 70 |
|
| 71 |
# Parse raw text (handles sentence splitting + tokenization)
|
|
|
|
| 77 |
print(f"{token.form:<15} {token.lemma:<15} {token.upos:<8} head={token.head} {token.deprel}")
|
| 78 |
```
|
| 79 |
|
| 80 |
+
Refer to the combo-nlp documentation for installation and usage instructions:
|
| 81 |
|
| 82 |
+
- [https://pypi.org/project/combo-nlp/](https://pypi.org/project/combo-nlp/)
|
| 83 |
+
- [https://pypi.org/project/combo-seg/](https://pypi.org/project/combo-seg/)
|
| 84 |
|
| 85 |
## License
|
| 86 |
|
|
|
|
| 110 |
|
| 111 |
## Resources
|
| 112 |
|
| 113 |
+
- combo-nlp: [https://pypi.org/project/combo-nlp/](https://pypi.org/project/combo-nlp/)
|
| 114 |
+
- combo-seg: [https://pypi.org/project/combo-seg/](https://pypi.org/project/combo-seg/)
|
| 115 |
- UD_Telugu-MTG: [https://github.com/UniversalDependencies/UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG)
|
| 116 |
|