SozKZ Vocab: Kazakh Tokenizers - a stukenov Collection

stukenov 's Collections

EkiTil: Bilingual Kazakh-Russian Language Models

SozKZ Misc: TTS, Sentiment & Other

SozKZ Vocab: Kazakh Tokenizers

SozKZ Corpora: Kazakh Training Datasets

SozKZ MoE: Mixture of Experts

SozKZ GEC: Kazakh Grammar Error Correction

SozKZ Core: Kazakh Language Models

SozKZ Vocab: Kazakh Tokenizers

updated Mar 25

BPE and SentencePiece tokenizers trained on Kazakh text — 32K vocabularies