---
license: apache-2.0
language:
- tr
pipeline_tag: text-generation
tags:
- slm
- base-model
- causal-lm
- pre-trained
- tr-llm
- Ahıska
- AhiskaTurks
- MeskhetianTurks
- AhıskaTürkleri
library_name: transformers

---

# AhiskaAI-65m-IT-v0.2

AhiskaAI-65m-IT-v0.2 is the instruction-tuned version of our 65M parameter Small Language Model. Fine-tuned on a curated Turkish instruction dataset, it is designed to function as a lightweight conversational AI assistant while maintaining fast inference on resource-constrained hardware.

**Base Model:** AhiskaAI-65m-Base-v0.2

---

## Model Details

- **Architecture:** Llama-based architecture.
- **Fine-tuning:** Supervised Fine-Tuning (SFT).
- **Format:** ChatML.
- **Parameters:** 65M.
- **Context Window:** 1024 tokens.
- **Tokenizer:** Custom BPE Tokenizer (Vocabulary Size: 32,000).
- **Training Framework:** PyTorch & Transformers.
- **Hardware:** NVIDIA RTX 4050 6GB Laptop GPU.

---

## Fine-tuning Dataset

The model was fine-tuned using a curated Turkish instruction dataset designed to improve conversational ability and instruction following.

The dataset focuses on:

- Question answering
- General conversation
- Summarization
- Text generation
- Turkish instruction following

---

## Design Goal

The 65M-IT model is designed as the lightweight conversational member of the AhiskaAI v0.2 family.

Its primary goals are:

- Basic Turkish instruction following.
- Fast conversational inference.
- Low-resource deployment.
- A compact research baseline for future alignment methods.

---

## Training Logs

![Training Loss Curve](training_loss.png)

*The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-65m-IT-v0.2.*

---

## Usage (ChatML)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-65m-IT-v0.2")
tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-65m-IT-v0.2")

SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın."

prompt = (
    f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n"
    f"<|im_start|>user\nMerhaba<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Known Limitations

- Limited factual knowledge due to model size.
- Optimized primarily for Turkish.
- Context window limited to 1024 tokens.
- May generate inaccurate or incomplete responses on complex topics.

---

## Future Plans

- DPO preference alignment.
- Improved instruction datasets.
- Future AhiskaAI v0.3 releases.

---

## About AhiskaAI

AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch.

Follow us on Hugging Face for updates and future releases.