Upload README.md with huggingface_hub

README.md

@@ -1,21 +1,125 @@
 ---
-base_model: unsloth/qwen2.5-7b-unsloth-bnb-4bit
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- qwen2
-license: apache-2.0
 language:
+- ta
 - en
+license: apache-2.0
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- tamil
+- qwen2
+- qlora
+- instruction-tuning
+- morphology
+- dravidian
+datasets:
+- Tamil-ai/samacheer-kalvi-tamil
+model-index:
+- name: Tamil-Qwen2.5-7B-Instruct
+  results: []
 ---
 
-# Uploaded finetuned  model
+# Tamil-Qwen2.5-7B-Instruct
+
+A Tamil-specialized instruction-tuned LLM built on Qwen2.5-7B-Instruct using QLoRA fine-tuning on 150K deduplicated Tamil instruction pairs.
+
+**Paper:** *"A Thousand Language Problem: Morphological Understanding in Linguistic AI"*
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Base model | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
+| Parameters | 7.6B |
+| Method | QLoRA (r=64, alpha=128, dropout=0.05) |
+| Training data | 150K deduplicated Tamil instruction-response pairs |
+| Tokenizer efficiency | 4.62x ratio (best among tested models for Tamil) |
+| Compute | RunPod RTX 5090, ~$5 total cost |
+| Sequence length | 1024 |
+| Batch size | 32 (effective) |
+| Epochs | 1 |
+
+## Training Data
+
+150,000 deduplicated instruction-response pairs from 5 Tamil datasets:
+- Tamil Alpaca
+- Tamil Orca
+- Tamil Dolly
+- Tamil-ai/samacheer-kalvi-tamil (morphological drills + grammar QA)
+- Additional Tamil instruction sets
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Tamil-ai/tamil-qwen25-7b-instruct"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype="auto",
+    device_map="auto",
+)
+
+messages = [
+    {"role": "system", "content": "You are a helpful Tamil language assistant."},
+    {"role": "user", "content": "வீடு என்ற சொல்லின் வேற்றுமை வடிவங்களைக் கூறுக."},
+]
+
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=256)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### 4-bit Quantized (for limited VRAM)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+model = AutoModelForCausalLM.from_pretrained(
+    "Tamil-ai/tamil-qwen25-7b-instruct",
+    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
+    device_map="auto",
+)
+```
+
+## Why Qwen2.5?
+
+Tokenizer analysis across 6 base models showed Qwen2.5 has the best Tamil tokenization efficiency:
+
+| Model | Tamil Token Ratio | Verdict |
+|-------|------------------|---------|
+| **Qwen2.5** | **4.62x** | Best for Tamil |
+| Llama 3.1 | 5.8x | |
+| Gemma 2 | 6.1x | |
+| Mistral | 7.2x | |
+| Falcon | 10.5x | Worst |
+
+Lower ratio = fewer tokens per Tamil word = more efficient training and inference.
+
+## Intended Use
+
+- Tamil question answering and instruction following
+- Tamil morphological analysis
+- Tamil grammar and linguistics tasks
+- Research on low-resource language LLMs
+
+## Limitations
 
-- **Developed by:** Tamil-ai
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/qwen2.5-7b-unsloth-bnb-4bit
+- Trained primarily on instructional Tamil; may underperform on colloquial/slang
+- Morphological accuracy varies by category (see benchmark results)
+- English capabilities may degrade compared to base Qwen2.5
 
-This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Citation
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+```bibtex
+@misc{tamilai2026,
+  title={A Thousand Language Problem: Morphological Understanding in Linguistic AI},
+  author={Tamil-AI},
+  year={2026},
+  publisher={HuggingFace},
+  url={https://huggingface.co/Tamil-ai/tamil-qwen25-7b-instruct}
+}
+```