| --- |
| license: other |
| language: |
| - en |
| - es |
| - fr |
| - de |
| - it |
| - pt |
| - ru |
| - ar |
| - hi |
| - ko |
| - zh |
| library_name: transformers |
| base_model: |
| - arcee-ai/Trinity-Nano-Preview |
| base_model_relation: quantized |
| license_link: LICENSE |
| license_name: openmdw-1.1 |
| --- |
| <div align="center"> |
| <picture> |
| <img |
| src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/i-v1KyAMOW_mgVGeic9WJ.png" |
| alt="Arcee Trinity Mini" |
| style="max-width: 100%; height: auto;" |
| > |
| </picture> |
| </div> |
| |
| # Trinity Nano Preview GGUF |
|
|
| Trinity Nano Preview is a preview of Arcee AI's 6B MoE model with 1B active parameters. It is the small-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike. |
|
|
| This is a chat tuned model, with a delightful personality and charm we think users will love. We note that this model is pushing the limits of sparsity in small language models with only 800M non-embedding parameters active per token, and as such **may be unstable** in certain use cases, especially in this preview. |
|
|
| This is an *experimental* release, it's fun to talk to but will not be hosted anywhere, so download it and try it out yourself! |
|
|
| These are the GGUF files for running on llama.cpp powered platforms |
|
|
| *** |
| |
| Trinity Nano Preview is trained on 10T tokens gathered and curated through a key partnership with [Datology](https://www.datologyai.com/), building upon the excellent dataset we used on [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) with additional math and code. |
| |
| Training was performed on a cluster of 512 H200 GPUs powered by [Prime Intellect](https://www.primeintellect.ai/) using HSDP parallelism. |
| |
| More details, including key architecture decisions, can be found on our blog [here](https://www.arcee.ai/blog/the-trinity-manifesto) |
| |
| *** |
|
|
| ## Model Details |
|
|
| * **Model Architecture:** AfmoeForCausalLM |
| * **Parameters:** 6B, 1B active |
| * **Experts:** 128 total, 8 active, 1 shared |
| * **Context length:** 128k |
| * **Training Tokens:** 10T |
| * **License:** [OpenMDW-1.1](https://huggingface.co/arcee-ai/Trinity-Mini#license) |
|
|
| *** |
| |
| <div align="center"> |
| <picture> |
| <img src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/sSVjGNHfrJKmQ6w8I18ek.png" style="background-color:ghostwhite;padding:5px;" width="17%" alt="Powered by Datology"> |
| </picture> |
| </div> |
| |
| ### Running our model |
| |
| - [llama.cpp](https://huggingface.co/arcee-ai/Trinity-Mini#llamacpp) |
| - [LM Studio](https://huggingface.co/arcee-ai/Trinity-Mini#lm-studio) |
| |
| ## llama.cpp |
| |
| Supported in llama.cpp release b7061 |
| |
| Download the latest [llama.cpp release](https://github.com/ggml-org/llama.cpp/releases) |
| |
| ``` |
| llama-server -hf arcee-ai/Trinity-Nano-Preview-GGUF:q4_k_m |
| ``` |
| |
| ## LM Studio |
| |
| Supported in latest LM Studio runtime |
| |
| Update to latest available, then verify your runtime by: |
| |
| 1. Click "Power User" at the bottom left |
| 2. Click the green "Developer" icon at the top left |
| 3. Select "LM Runtimes" at the top |
| 4. Refresh the list of runtimes and verify that the latest is installed |
| |
| Then, go to Model Search and search for `arcee-ai/Trinity-Nano-Preview-GGUF`, download your prefered size, and load it up in the chat |
| |
| |
| ## License |
| |
| Trinity-Nano-Preview is released under the OpenMDW-1.1 license. |
