Text Generation
Transformers
Ukrainian
text2text-generation
skypro1111 commited on
Commit
559327e
·
verified ·
1 Parent(s): 882b4b5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +99 -0
README.md CHANGED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - skypro1111/ubertext-2-news-verbalized
5
+ language:
6
+ - uk
7
+ base_model:
8
+ - skypro1111/m2m100-ukr-verbalization
9
+ pipeline_tag: text2text-generation
10
+ library_name: transformers
11
+ widget:
12
+ - text: >-
13
+ Очікувалось, що цей застосунок буде запущено о 11 ранку 22.08.2025, але
14
+ розробники затягнули святкування і запуск був відкладений на 2 тижні.
15
+ ---
16
+
17
+ # m2m100-ukr-verbalization-ct2
18
+
19
+ This is the CTranslate2-optimized version of [skypro1111/m2m100-ukr-verbalization](https://huggingface.co/skypro1111/m2m100-ukr-verbalization).
20
+
21
+ ## About CTranslate2 Optimization
22
+
23
+ CTranslate2 is an optimized inference engine for Transformer models that provides much faster inference than standard implementations. This repository contains the m2m100-418M model for Ukrainian verbalization, optimized for production deployment.
24
+
25
+ Key advantages:
26
+ - **Faster inference**: 3-5x speed improvement over the standard Transformers implementation
27
+ - **Lower memory footprint**: Uses efficient int8/float16 quantization
28
+ - **Better hardware utilization**: Optimized for both CPU and GPU
29
+ - **Efficient batching**: Improved batch processing for production workloads
30
+
31
+ ## Usage
32
+
33
+ ```python
34
+ import os
35
+ import ctranslate2
36
+ from transformers import M2M100Tokenizer
37
+
38
+ def process_sentence(translator, tokenizer, sentence: str):
39
+ # Tokenize input
40
+ source = tokenizer.convert_ids_to_tokens(tokenizer.encode(sentence))
41
+ target_prefix = [tokenizer.lang_code_to_token["uk"]]
42
+
43
+ # Run inference
44
+ results = translator.translate_batch(
45
+ [source],
46
+ target_prefix=[target_prefix],
47
+ beam_size=1,
48
+ num_hypotheses=1,
49
+ use_vmap=True,
50
+ )
51
+
52
+ # Get target tokens and decode
53
+ target = results[0].hypotheses[0][1:] # Remove language token
54
+ return tokenizer.decode(tokenizer.convert_tokens_to_ids(target))
55
+
56
+ # Initialize translator with optimizations
57
+ translator = ctranslate2.Translator(
58
+ "path/to/m2m100-ukr-verbalization-ct2",
59
+ device="cuda",
60
+ compute_type="int8_float16", # Options: int8, int8_float16, float16, float32
61
+ intra_threads=16,
62
+ )
63
+
64
+ # Load tokenizer from the original model
65
+ tokenizer = M2M100Tokenizer.from_pretrained("skypro1111/m2m100-ukr-verbalization")
66
+ tokenizer.src_lang = "uk"
67
+
68
+ # Example
69
+ text = "Моя бабуся народилася 07.11.1919, у важкий післявоєнний час."
70
+ result = process_sentence(translator, tokenizer, text)
71
+ print(result)
72
+ # Output: "Моя бабуся народилася сьомого листопада тисяча дев'ятсот дев'ятнадцятого року, у важкий післявоєнний час."
73
+ ```
74
+
75
+ ## Performance Comparison
76
+
77
+ | Metric | Original Model | CTranslate2 Model |
78
+ |--------|---------------|-------------------|
79
+ | Inference time (avg) | ~300ms | ~70ms |
80
+ | Memory usage | ~2.5GB | ~900MB |
81
+ | Batch processing (10) | ~2.8s | ~0.7s |
82
+
83
+ *Measurements taken on NVIDIA T4 GPU with int8_float16 quantization
84
+
85
+ ## Integration
86
+
87
+ This model can be easily integrated into production systems using:
88
+ - Python with CTranslate2 library
89
+ - C++ applications via CTranslate2 C++ API
90
+ - REST API services
91
+ - Docker containers
92
+
93
+ ## Model Information
94
+
95
+ For detailed information about the model capabilities, training data, and usage examples, please refer to the original model: [skypro1111/m2m100-ukr-verbalization](https://huggingface.co/skypro1111/m2m100-ukr-verbalization)
96
+
97
+ ## License
98
+
99
+ This model is released under the MIT License, in line with the original m2m100-ukr-verbalization model.