---
license: gemma
base_model: unsloth/gemma-3-1b-it
tags:
- tamil
- tamil-nadu
- cultural-ai
- gemma
- fine-tuned
language:
- en
- ta
datasets:
- nvidia/Nemotron-Personas-India
library_name: transformers
pipeline_tag: text-generation
widget:
- text: "Tell me about Pongal festival"
- text: "Vanakkam! How are you?"
extra_gated_heading: "Acknowledge license to accept the repository"
extra_gated_button_content: "Acknowledge license"
repo_url: https://github.com/Naazimsnh02/gemma-3-tamilnadu_sample
---

# Gemma-3-1B Tamilnadu Sample

[![Model on HF](https://huggingface.co/datasets/huggingface/badges/resolve/main/model-on-hf-md.svg)](https://huggingface.co/naazimsnh02/gemma-3-tamilnadu_sample)
[![GitHub](https://img.shields.io/badge/GitHub-Repository-black?logo=github)](https://github.com/Naazimsnh02/gemma-3-tamilnadu_sample)

## Model Description

This model is a fine-tuned version of **Gemma-3-1B-IT** specifically trained on **Tamil Nadu personas and cultural context**. 
It understands and responds with authentic Tamil Nadu cultural knowledge, mixing Tamil and English naturally, and 
demonstrates deep understanding of Tamil traditions, festivals, language, and regional nuances.

### Key Features

- 🎯 **Tamil Nadu Focused**: Trained exclusively on personas from Tamil Nadu
- 🗣️ **Bilingual**: Natural code-mixing of Tamil and English
- 🎭 **Cultural Awareness**: Deep understanding of Tamil festivals, traditions, and customs
- 🛡️ **Safety Aligned**: Includes safety fine-tuning to refuse harmful requests
- 📚 **Multi-Stage Training**: SFT → Instruction Tuning → DPO → Safety

## Training Details

### Training Data

- **Primary Dataset**: Nemotron-Personas-India (en_IN split, Tamil Nadu only)
- **Personas**: ~20,000 Tamil Nadu personas
- **Instructions**: Tamil Nadu-specific + general knowledge mix
- **DPO Pairs**: Cultural preference alignment
- **Safety Examples**: Tamil Nadu context-aware safety responses

### Training Stages

1. **Stage 1 - Supervised Fine-Tuning (SFT)**
   - Dataset: Tamil Nadu personas from Nemotron-Personas-India
   - Steps: 3000
   - Focus: Learning to roleplay as diverse Tamil Nadu personas

2. **Stage 2 - Instruction Tuning**
   - Dataset: Tamil Nadu instructions + general knowledge
   - Steps: 500
   - Mix Ratio: 80% Tamil Nadu, 20% General

3. **Stage 3 - DPO (Direct Preference Optimization)**
   - Dataset: Tamil Nadu preference pairs
   - Steps: 150
   - Beta: 0.1
   - Focus: Aligning responses with Tamil cultural preferences

4. **Stage 4 - Safety Fine-Tuning**
   - Dataset: Safety examples with Tamil Nadu context
   - Steps: 50
   - Focus: Ethical responses, anti-discrimination, cultural sensitivity

### Training Configuration

- **Base Model**: unsloth/gemma-3-1b-it
- **LoRA Rank**: 32
- **LoRA Alpha**: 32
- **Max Sequence Length**: 2048
- **Quantization**: 4-bit
- **Training Framework**: Unsloth + TRL

## Usage

### Basic Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "naazimsnh02/gemma-3-tamilnadu_sample"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Example prompt
messages = [
    {"role": "user", "content": "Tell me about Pongal festival in Tamil Nadu"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

### With Unsloth (Faster Inference)

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="naazimsnh02/gemma-3-tamilnadu_sample",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(model)

# Generate response
messages = [{"role": "user", "content": "Vanakkam! How are you?"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Limitations

- **Language**: Primarily English with Tamil code-mixing; not fluent in pure Tamil text generation
- **Geographic Scope**: Optimized for Tamil Nadu context; may not generalize well to other Indian states
- **Persona Bias**: Training focused on specific demographic distributions from the dataset
- **Cultural Nuances**: May not capture all sub-regional variations within Tamil Nadu
- **Safety**: While safety-tuned, may still generate inappropriate content in edge cases

## Ethical Considerations

- **Anti-Discrimination**: Model is trained to reject caste-based, religious, or regional discrimination
- **Cultural Sensitivity**: Respects Tamil Nadu's diverse communities and traditions
- **Bias Mitigation**: Includes safety fine-tuning to promote equality and respect
- **Responsible Use**: Should not be used to generate harmful, discriminatory, or misleading content

## Training Infrastructure

- **GPU**: NVIDIA T4 (Google Colab)
- **Training Time**: ~10 hours (full pipeline)
- **Framework**: Unsloth + Transformers + TRL
- **Optimization**: 4-bit quantization, LoRA adapters, gradient checkpointing

## Citation

If you use this model, please cite:

```bibtex
@misc{tamil-nadu-gemma-2025,
  title={Tamil Nadu Cultural AI Model based on Gemma-3-1B-IT},
  author={Syed Naazim Hussain},
  year={2025},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/naazimsnh02/gemma-3-tamilnadu_sample}}
}
```

## Acknowledgments

- **Base Model**: Google's Gemma-3-1B-IT via Unsloth
- **Dataset**: NVIDIA's Nemotron-Personas-India
- **Framework**: Unsloth for efficient training
- **Community**: Tamil Nadu's rich cultural heritage

## License

This model inherits the Gemma license from the base model. Please review the license terms before use.

## Contact

For questions, issues, or feedback, please open an issue on the model repository.

---

**Note**: This model is designed for research and educational purposes. Always verify outputs for accuracy and cultural appropriateness.