File size: 1,677 Bytes
93343f2 4b46f35 39e3da9 93343f2 933fdd9 93343f2 2da7fd1 93343f2 ad112fd 93343f2 dbb6a31 93343f2 b2f8793 93343f2 dbb6a31 93343f2 dbb6a31 93343f2 dbb6a31 93343f2 dbb6a31 93343f2 8e5f0ef 93343f2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | ---
license: mit
language:
- en
base_model:
- meta-llama/Llama-3.1-8B-Instruct
tags:
- axolotl
datasets:
- NarrativAI/CakrawalaRP
pipeline_tag: text-generation
---
# 🎭 Cakrawala-Llama-3.1-8B
> *Where Worlds Converge and Adventures Begin!*
## 🌟 What's Special About This Model?
Cakrawala-Llama-3.1-8B is a fine-tuned variant of the Llama-3.1-8B-Instruct model, specifically optimised for generating rich roleplaying conversations and character interactions. The model has been trained to excel at producing detailed, contextually appropriate character dialogues with rich descriptions of physical actions, expressions, and emotional states while maintaining consistent character voices and perspectives throughout extended interactions.
## 🧪 The Secret Sauce
### Training Diet:
- Fed with 13,000 conversation pairs
- Each conversation is a minimum 12-13 turns long
- Focused heavily details like facial expressions, environmental descriptions, and character reactions that are focused a lot on **keeping the model in character.**
### Tech Wizardry:
- Trained on Llama-3.1-8B-Instruct
- Fine-tuned using QLoRA
- Trained over 2 epochs
## Training Parameters
- Gradient Accumulation Steps: 1
- Micro Batch Size: 4
- Learning Rate: 0.0002
- Optimizer: AdamW
- Scheduler: Cosine
- Mixed Precision: BF16 & FP16 with TF32 support
## 🔧 Under the Hood
- Trained on 8 x H100 NVL GPUs
## 🎬 License & Credits
- Licensed under MIT
- Based on meta-llama/Llama-3.1-8B-Instruct
---
## GGUF Quants
- https://huggingface.co/mradermacher/Cakrawala-8B-GGUF
- https://huggingface.co/mradermacher/Cakrawala-8B-i1-GGUF
*Built with ❤️ for roleplayers, by roleplayers* |