Hugging Face's logo

clarin-pl
/
combo-seg-xlm-roberta-base-telugu-mtg-ud2.17

Token Classification
PyTorch
Telugu
segmentation
tokenization
combo-seg
universal-dependencies
Model card Files
xet
 Community
combo-seg-xlm-roberta-base-telugu-mtg-ud2.17
File size: 3,081 Bytes
8ca297f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8db8799
 
9e8923f
8ca297f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8db8799
8ca297f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8db8799
8ca297f
 
 
 
 
 
 
 
 
 
 
 
 
8db8799
8ca297f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
---
license: cc-by-sa-4.0
language:
  - te
tags:
  - segmentation
  - tokenization
  - combo-seg
  - universal-dependencies
datasets:
  - universal_dependencies
model-name: combo-seg-xlm-roberta-base-telugu-mtg-ud2.17
pipeline_tag: token-classification
---

# COMBO-SEG Model for Telugu

## Model Description

This is a Telugu-language character-level segmentation model based on [COMBO-SEG](https://gitlab.clarin-pl.eu/syntactic-tools/combo-seg), an open-source text segmentation system. It performs:

- sentence segmentation
- tokenisation (including multi-word token detection)

The Telugu model uses ``FacebookAI/xlm-roberta-base`` as its base encoder and is trained on [UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG) (UD v2.17).


## Evaluation

| Metric | Tokens | Words | Sentences |
| ------ | ------ | ----- | --------- |
| F1 | 99.79 | 99.79 | 97.58 |


## Usage

Install the library from PyPI:

```bash
pip install combo-seg
```

```python
from combo_seg import ComboSeg

# Load a pre-trained model
nlp = ComboSeg("Telugu")

# Segment raw text — returns Document with hierarchy: Document -> Turn -> Sentence -> Token
doc = nlp("వేగవంతమైన గోధుమ రంగు నక్క బద్ధకపు కుక్క మీదుగా దూకుతుంది.")

# Inspect results
for turn in doc.turns:
    for sentence in turn.sentences:
        print(f"Sentence: {sentence.text}")
        for token in sentence.tokens:
            if token.is_multi_word:
                print(f"  MWT: {token.text} -> {token.subwords}")
            else:
                print(f"  Token: {token.text}")
```

Or load directly from HuggingFace:

```python
from combo_seg import ComboSeg

nlp = ComboSeg.from_pretrained("clarin-pl/combo-seg-xlm-roberta-base-telugu-mtg-ud2.17")
doc = nlp("వేగవంతమైన గోధుమ రంగు నక్క బద్ధకపు కుక్క మీదుగా దూకుతుంది.")
```

## License

The training data license: cc-by-sa-4.0 is derived from the Universal Dependencies treebank. For the full license terms of each treebank, please refer to the corresponding `LICENSE.txt` file in the treebank repository:

- [UD_Telugu-MTG LICENSE.txt](https://github.com/UniversalDependencies/UD_Telugu-MTG/blob/master/LICENSE.txt)


## Citation

If you use this model, please cite:

Ulewicz, M., & Wróblewska, A. (2026). *COMBO-SEG Models Trained on UD v2.17*. https://doi.org/10.5281/zenodo.19651441

```bibtex
@software{combo_seg_2026,
  author    = {Ulewicz, Michał and Wróblewska, Alina},
  title     = {{COMBO-SEG} Models Trained on {UD} v2.17},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19651441},
  url       = {https://doi.org/10.5281/zenodo.19651441}
}
```

## Resources

- COMBO-SEG: [https://gitlab.clarin-pl.eu/syntactic-tools/combo-seg](https://gitlab.clarin-pl.eu/syntactic-tools/combo-seg)
- UD_Telugu-MTG: [https://github.com/UniversalDependencies/UD_Telugu-MTG](https://github.com/UniversalDependencies/UD_Telugu-MTG)