--- license: cc-by-sa-4.0 language: - te tags: - dependency-parsing - combo - universal-dependencies datasets: - universal_dependencies model-name: Combo Nlp Xlm Roberta Base Telugu Mtg Ud2.17 pipeline_tag: token-classification --- # COMBO-NLP Model for Telugu ## Model Description This is a Telugu-language model based on [COMBO-NLP](https://gitlab.clarin-pl.eu/syntactic-tools/combo-nlp), an open-source natural language preprocessing system. It performs: - sentence segmentation (via [LAMBO](https://gitlab.clarin-pl.eu/syntactic-tools/lambo)) - tokenisation (via [LAMBO](https://gitlab.clarin-pl.eu/syntactic-tools/lambo)) - part-of-speech tagging - morphological analysis - lemmatisation - dependency parsing The Telugu model uses ``FacebookAI/xlm-roberta-base`` as its base encoder and is trained on [UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG) (UD v2.17). ## Evaluation Evaluation was performed on the UD_Telugu-MTG test split using the standard [CoNLL 2018 eval script](https://universaldependencies.org/conll18/conll18_ud_eval.py). Two evaluation rows are reported: - **Full-text (F1)**: raw text is segmented by [LAMBO](https://gitlab.clarin-pl.eu/syntactic-tools/lambo), then parsed and compared against gold — measures end-to-end pipeline performance including segmentation quality. - **Aligned accuracy**: accuracy on correctly segmented (aligned) tokens — measures parsing quality on tokens that were correctly identified by the segmenter. ### Morphosyntactic Tagging | Metric | Tokens | Sentences | Words | UPOS | XPOS | UFeats | AllTags | Lemmas | | ------ | ------ | --------- | ----- | ---- | ---- | ------ | ------- | ------ | | Full-text (F1) | 99.79 | 96.60 | 99.79 | 94.94 | 94.80 | 98.96 | 94.66 | 99.79 | | Aligned accuracy | 0.00 | 0.00 | 0.00 | 95.14 | 95.00 | 99.17 | 94.86 | 100.00 | ### Dependency Parsing | Metric | UAS | LAS | CLAS | MLAS | BLEX | | ------ | --- | --- | ---- | ---- | ---- | | Full-text (F1) | 91.61 | 84.13 | 80.76 | 77.33 | 80.76 | | Aligned accuracy | 91.81 | 84.31 | 80.92 | 77.48 | 80.92 | ## Usage Install the library from PyPI (assuming you have a virtual environment created): ```bash pip install combo-nlp ``` Install the Lambo segmenter - only needed when passing raw text strings to COMBO: ```bash pip install --index-url https://pypi.clarin-pl.eu/ lambo ``` ```python from combo import COMBO # Load a pre-trained model with corresponding Lambo segmenter nlp = COMBO("Telugu") # Parse raw text (handles sentence splitting + tokenization) result = nlp("వేగవంతమైన గోధుమ రంగు నక్క సోమరి కుక్క మీదుగా దూకుతుంది.") # Inspect results for sentence in result: for token in sentence: print(f"{token.form:<15} {token.lemma:<15} {token.upos:<8} head={token.head} {token.deprel}") ``` Refer to the COMBO-NLP documentation for installation and usage instructions: - [https://gitlab.clarin-pl.eu/syntactic-tools/combo-nlp](https://gitlab.clarin-pl.eu/syntactic-tools/combo-nlp) - [https://gitlab.clarin-pl.eu/syntactic-tools/lambo](https://gitlab.clarin-pl.eu/syntactic-tools/lambo) ## Citation ## Resources - COMBO-NLP: [https://gitlab.clarin-pl.eu/syntactic-tools/combo-nlp](https://gitlab.clarin-pl.eu/syntactic-tools/combo-nlp) - LAMBO: [https://gitlab.clarin-pl.eu/syntactic-tools/lambo](https://gitlab.clarin-pl.eu/syntactic-tools/lambo) - UD_Telugu-MTG: [https://github.com/UniversalDependencies/UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG)