Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers

Commit History

Upload README.md with huggingface_hub
fed6fc1
verified

Ouaill commited on

Upload external_comparison.csv with huggingface_hub
4ecd5e0
verified

Ouaill commited on

Upload test_set_results.csv with huggingface_hub
7ff9050
verified

Ouaill commited on

Upload tokenizers/concat_az_bbpe_55000.json with huggingface_hub
6271d4a
verified

Ouaill commited on

Upload tokenizers/concat_ar_bbpe_55000.json with huggingface_hub
b0d9588
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_55000.json with huggingface_hub
473b885
verified

Ouaill commited on

Upload tokenizers/concat_ar_wordpiece_55000.json with huggingface_hub
45b9fbf
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_55000.json with huggingface_hub
d6074e5
verified

Ouaill commited on

Upload tokenizers/concat_ar_unigram_55000.json with huggingface_hub
be8d5ce
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_55000.json with huggingface_hub
c844c8b
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_55000.json with huggingface_hub
8f9aec4
verified

Ouaill commited on

Upload tokenizers/concat_az_bbpe_40000.json with huggingface_hub
c91806d
verified

Ouaill commited on

Upload tokenizers/concat_ar_bbpe_40000.json with huggingface_hub
c13d967
verified

Ouaill commited on

Upload tokenizers/concat_az_wordpiece_40000.json with huggingface_hub
ceddd9d
verified

Ouaill commited on

Upload tokenizers/concat_ar_wordpiece_40000.json with huggingface_hub
317fc7b
verified

Ouaill commited on

Upload tokenizers/concat_az_unigram_40000.json with huggingface_hub
bb54448
verified

Ouaill commited on

Upload tokenizers/concat_ar_unigram_40000.json with huggingface_hub
575deaf
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_40000.json with huggingface_hub
c76c893
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_40000.json with huggingface_hub
a1760cc
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_110000.json with huggingface_hub
b52a919
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_110000.json with huggingface_hub
a3a9d89
verified

Ouaill commited on

Upload tokenizers/shared_unigram_110000.json with huggingface_hub
52db63a
verified

Ouaill commited on

Upload tokenizers/shared_bpe_110000.json with huggingface_hub
458c81b
verified

Ouaill commited on

Upload tokenizers/shared_bbpe_80000.json with huggingface_hub
9d726e7
verified

Ouaill commited on

Upload tokenizers/shared_wordpiece_80000.json with huggingface_hub
d467a27
verified

Ouaill commited on

Upload tokenizers/shared_unigram_80000.json with huggingface_hub
b586f2b
verified

Ouaill commited on

Upload tokenizers/shared_bpe_80000.json with huggingface_hub
81a1702
verified

Ouaill commited on

Upload tokenizers/concat_az_bpe_16000.json with huggingface_hub
a2b3e8d
verified

Ouaill commited on

Upload tokenizers/concat_ar_bpe_16000.json with huggingface_hub
476c81a
verified

Ouaill commited on

Upload tokenizer_results.csv with huggingface_hub
578f16a
verified

Ouaill commited on

Upload external_comparison.csv with huggingface_hub
1e6beec
verified

Ouaill commited on

Upload doda_independent_results.csv with huggingface_hub
bdf5a6e
verified

Ouaill commited on

Upload codeswitch_results.csv with huggingface_hub
ec48621
verified

Ouaill commited on

Upload bootstrap_ci_test_set.csv with huggingface_hub
9793569
verified

Ouaill commited on

Upload test_set_results.csv with huggingface_hub
9715656
verified

Ouaill commited on

Upload plots/fertility_overall_faceted.png with huggingface_hub
6ee779e
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_comparison_v2.png with huggingface_hub
79e19cc
verified

Ouaill commited on

Upload plots/cpt_overall_comparison_v2.png with huggingface_hub
13fbc56
verified

Ouaill commited on

Upload plots/exact_match_rate_comparison_v2.png with huggingface_hub
75e957e
verified

Ouaill commited on

Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub
5d31685
verified

Ouaill commited on

Upload plots/fertility_disparity_trends.png with huggingface_hub
83c46df
verified

Ouaill commited on

Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub
7059b0c
verified

Ouaill commited on

Upload plots/morph_consistency_f1_comparison_v2.png with huggingface_hub
bfa6f98
verified

Ouaill commited on

Upload plots/morph_consistency_f1_heatmap_v2.png with huggingface_hub
9eb984a
verified

Ouaill commited on

Upload plots/fertility_overall_comparison_v2.png with huggingface_hub
79fac54
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_heatmap_v2.png with huggingface_hub
d760658
verified

Ouaill commited on

Upload plots/morph_edit_distance_ar_trends.png with huggingface_hub
0b5bb9d
verified

Ouaill commited on

Upload plots/exact_match_rate_heatmap_v2.png with huggingface_hub
77fbb17
verified

Ouaill commited on

Upload plots/cpt_overall_trends.png with huggingface_hub
7338ed2
verified

Ouaill commited on

Upload plots/fertility_overall_heatmap_v2.png with huggingface_hub
d465e60
verified

Ouaill commited on