Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,7 @@ tags:
|
|
| 11 |
- qwen
|
| 12 |
- qaraqalpaq
|
| 13 |
- alpaca
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
# Qwen2.5-3B-Karakalpak-Base (Checkpoint 3000)
|
|
@@ -23,8 +24,20 @@ This is a **Continued Pre-training (CPT)** adapter.
|
|
| 23 |
- **Behavior:** This model is designed for **text completion**. It is **NOT** instruction-tuned. It will not respond to "Chat" prompts (e.g., "Tell me a story") reliably yet.
|
| 24 |
- **Next Steps:** This serves as the foundation for future instruction-tuning (SFT).
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
## Model Details
|
| 27 |
- **Developed by:** [kdrnyzv890](https://huggingface.co/kdrnyzv890)
|
|
|
|
| 28 |
- **Training Data:** 150MB of raw Karakalpak text (news, literature, web content).
|
| 29 |
- **Training Depth:** ~0.3 Epochs (Checkpoint 3000).
|
| 30 |
- **Fine-tuning Technique:** LoRA (Low-Rank Adaptation).
|
|
|
|
| 11 |
- qwen
|
| 12 |
- qaraqalpaq
|
| 13 |
- alpaca
|
| 14 |
+
- central-asia
|
| 15 |
---
|
| 16 |
|
| 17 |
# Qwen2.5-3B-Karakalpak-Base (Checkpoint 3000)
|
|
|
|
| 24 |
- **Behavior:** This model is designed for **text completion**. It is **NOT** instruction-tuned. It will not respond to "Chat" prompts (e.g., "Tell me a story") reliably yet.
|
| 25 |
- **Next Steps:** This serves as the foundation for future instruction-tuning (SFT).
|
| 26 |
|
| 27 |
+
## 🛑 Support & Collaboration
|
| 28 |
+
Training on this project was paused at **Checkpoint 3000** due to a lack of computational budget. As an independent developer working on a low-resource language, the costs of GPU power have outpaced my personal resources. To reach a full 1.0+ Epoch and proceed to the Instruction Tuning (Chat) phase, I am seeking support from the community.
|
| 29 |
+
|
| 30 |
+
**How you can help:**
|
| 31 |
+
* **Computational Power:** If you have spare GPU credits (H100/A100/A10G) or access to a compute cluster and would like to sponsor the next training run.
|
| 32 |
+
* **Data Contribution:** Cleaned Karakalpak datasets, specifically instruction-response pairs, are vital for the next phase.
|
| 33 |
+
* **Open Source Support:** Star this repository and share it with NLP researchers or within the Central Asian tech community to help find potential partners.
|
| 34 |
+
* **Development:** If you are an expert in SFT or RLHF and want to collaborate on making **"Bawir AI"** more capable.
|
| 35 |
+
|
| 36 |
+
---
|
| 37 |
+
|
| 38 |
## Model Details
|
| 39 |
- **Developed by:** [kdrnyzv890](https://huggingface.co/kdrnyzv890)
|
| 40 |
+
- **Project Name:** Bawir AI
|
| 41 |
- **Training Data:** 150MB of raw Karakalpak text (news, literature, web content).
|
| 42 |
- **Training Depth:** ~0.3 Epochs (Checkpoint 3000).
|
| 43 |
- **Fine-tuning Technique:** LoRA (Low-Rank Adaptation).
|