--- base_model: CohereForAI/aya-23-8B language: - en - fr - de - es - it - pt - ja - ko - zh - ar - el - fa - pl - id - cs - he - hi - nl - ro - ru - tr - uk - vi library_name: transformers license: cc-by-nc-4.0 tags: - llama-cpp - matrixportal inference: false extra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license) and acknowledge that the information you provide will be collected, used, and shared in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll receive email updates about C4AI and Cohere research, events, products and services. You can unsubscribe at any time. extra_gated_fields: Name: text Affiliation: text Country: type: select options: - Aruba - Afghanistan - Angola - Anguilla - Åland Islands - Albania - Andorra - United Arab Emirates - Argentina - Armenia - American Samoa - Antarctica - French Southern Territories - Antigua and Barbuda - Australia - Austria - Azerbaijan - Burundi - Belgium - Benin - Bonaire Sint Eustatius and Saba - Burkina Faso - Bangladesh - Bulgaria - Bahrain - Bahamas - Bosnia and Herzegovina - Saint Barthélemy - Belarus - Belize - Bermuda - Plurinational State of Bolivia - Brazil - Barbados - Brunei-Darussalam - Bhutan - Bouvet-Island - Botswana - Central African Republic - Canada - Cocos (Keeling) Islands - Switzerland - Chile - China - Côte-dIvoire - Cameroon - Democratic Republic of the Congo - Cook Islands - Colombia - Comoros - Cabo Verde - Costa Rica - Cuba - Curaçao - Christmas Island - Cayman Islands - Cyprus - Czechia - Germany - Djibouti - Dominica - Denmark - Dominican Republic - Algeria - Ecuador - Egypt - Eritrea - Western Sahara - Spain - Estonia - Ethiopia - Finland - Fiji - Falkland Islands (Malvinas) - France - Faroe Islands - Federated States of Micronesia - Gabon - United Kingdom - Georgia - Guernsey - Ghana - Gibraltar - Guinea - Guadeloupe - Gambia - Guinea Bissau - Equatorial Guinea - Greece - Grenada - Greenland - Guatemala - French Guiana - Guam - Guyana - Hong Kong - Heard Island and McDonald Islands - Honduras - Croatia - Haiti - Hungary - Indonesia - Isle of Man - India - British Indian Ocean Territory - Ireland - Islamic Republic of Iran - Iraq - Iceland - Israel - Italy - Jamaica - Jersey - Jordan - Japan - Kazakhstan - Kenya - Kyrgyzstan - Cambodia - Kiribati - Saint-Kitts-and-Nevis - South Korea - Kuwait - Lao-Peoples-Democratic-Republic - Lebanon - Liberia - Libya - Saint-Lucia - Liechtenstein - Sri Lanka - Lesotho - Lithuania - Luxembourg - Latvia - Macao - Saint Martin (French-part) - Morocco - Monaco - Republic of Moldova - Madagascar - Maldives - Mexico - Marshall Islands - North Macedonia - Mali - Malta - Myanmar - Montenegro - Mongolia - Northern Mariana Islands - Mozambique - Mauritania - Montserrat - Martinique - Mauritius - Malawi - Malaysia - Mayotte - Namibia - New Caledonia - Niger - Norfolk Island - Nigeria - Nicaragua - Niue - Netherlands - Norway - Nepal - Nauru - New Zealand - Oman - Pakistan - Panama - Pitcairn - Peru - Philippines - Palau - Papua New Guinea - Poland - Puerto Rico - North Korea - Portugal - Paraguay - State of Palestine - French Polynesia - Qatar - Réunion - Romania - Russia - Rwanda - Saudi Arabia - Sudan - Senegal - Singapore - South Georgia and the South Sandwich Islands - Saint Helena Ascension and Tristan da Cunha - Svalbard and Jan Mayen - Solomon Islands - Sierra Leone - El Salvador - San Marino - Somalia - Saint Pierre and Miquelon - Serbia - South Sudan - Sao Tome and Principe - Suriname - Slovakia - Slovenia - Sweden - Eswatini - Sint Maarten (Dutch-part) - Seychelles - Syrian Arab Republic - Turks and Caicos Islands - Chad - Togo - Thailand - Tajikistan - Tokelau - Turkmenistan - Timor Leste - Tonga - Trinidad and Tobago - Tunisia - Turkey - Tuvalu - Taiwan - United Republic of Tanzania - Uganda - Ukraine - United States Minor Outlying Islands - Uruguay - United-States - Uzbekistan - Holy See (Vatican City State) - Saint Vincent and the Grenadines - Bolivarian Republic of Venezuela - Virgin Islands British - Virgin Islands U.S. - VietNam - Vanuatu - Wallis and Futuna - Samoa - Yemen - South Africa - Zambia - Zimbabwe I agree to use this model for non-commercial use ONLY: checkbox --- # matrixportal/aya-23-8B-GGUF This model was converted to GGUF format from [`CohereForAI/aya-23-8B`](https://huggingface.co/CohereForAI/aya-23-8B) using llama.cpp via the ggml.ai's [all-gguf-same-where](https://huggingface.co/spaces/matrixportal/all-gguf-same-where) space. Refer to the [original model card](https://huggingface.co/CohereForAI/aya-23-8B) for more details on the model. ## ✅ Quantized Models Download List **✨ Recommended for CPU:** `Q4_K_M` | **⚡ Recommended for ARM CPU:** `Q4_0` | **🏆 Best Quality:** `Q8_0` | 🚀 Download | 🔢 Type | 📝 Notes | |:---------|:-----|:------| | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q2_k.gguf) | ![Q2_K](https://img.shields.io/badge/Q2_K-1A73E8) | Basic quantization | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q3_k_s.gguf) | ![Q3_K_S](https://img.shields.io/badge/Q3_K_S-34A853) | Small size | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q3_k_m.gguf) | ![Q3_K_M](https://img.shields.io/badge/Q3_K_M-FBBC05) | Balanced quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q3_k_l.gguf) | ![Q3_K_L](https://img.shields.io/badge/Q3_K_L-4285F4) | Better quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q4_0.gguf) | ![Q4_0](https://img.shields.io/badge/Q4_0-EA4335) | Fast on ARM | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q4_k_s.gguf) | ![Q4_K_S](https://img.shields.io/badge/Q4_K_S-673AB7) | Fast, recommended | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q4_k_m.gguf) | ![Q4_K_M](https://img.shields.io/badge/Q4_K_M-673AB7) ⭐ | Best balance | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q5_0.gguf) | ![Q5_0](https://img.shields.io/badge/Q5_0-FF6D01) | Good quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q5_k_s.gguf) | ![Q5_K_S](https://img.shields.io/badge/Q5_K_S-0F9D58) | Balanced | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q5_k_m.gguf) | ![Q5_K_M](https://img.shields.io/badge/Q5_K_M-0F9D58) | High quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q6_k.gguf) | ![Q6_K](https://img.shields.io/badge/Q6_K-4285F4) 🏆 | Very good quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-q8_0.gguf) | ![Q8_0](https://img.shields.io/badge/Q8_0-EA4335) ⚡ | Fast, best quality | | [Download](https://huggingface.co/matrixportal/aya-23-8B-GGUF/resolve/main/aya-23-8b-f16.gguf) | ![F16](https://img.shields.io/badge/F16-000000) | Maximum accuracy | 💡 **Tip:** Use `F16` for maximum precision when quality is critical