Skshackster/gemma3-270m-mental-health-fine-tuned-gguf

Gemma-3 270M Mental Health Fine-tuned Model

Model Description

This model is a fine-tuned version of Google's Gemma-3 270M, specifically trained for mental health conversational support using Cognitive Behavioral Therapy (CBT) principles. The model has been trained on 5M+ tokens of high-quality mental health conversational data to provide empathetic, supportive, and therapeutically-informed responses.

Developed by: Saurav Kumar Srivastava

Model Details

• Base Model: google/gemma-3-270m
• Model Size: 270M parameters
• Training Data: 5M+ tokens of CBT-based therapeutic conversations
• Training Method: LoRA fine-tuning using Unsloth
• Quantization: BF16 GGUF format available
• License: MIT

Training Configuration

The model was fine-tuned using the following specifications:

• LoRA Rank (r): 8
• LoRA Alpha: 8
• Target Modules: All attention and MLP modules
• Batch Size: 2 (per device) with 4 gradient accumulation steps
• Learning Rate: 2e-4
• Training Steps: 30 (optimized for efficiency)
• Optimizer: AdamW 8-bit
• Framework: Unsloth + TRL SFTTrainer

Intended Use

Primary Use Cases

• Mental Health Support: Providing empathetic conversations and CBT-based guidance
• Therapeutic Assistance: Supporting individuals with anxiety, depression, and stress management
• Educational Tool: Teaching CBT techniques and mental health awareness
• Research: Studying conversational AI in mental health applications

Limitations

• Not a Replacement for Professional Help: This model should not replace licensed mental health professionals
• Crisis Situations: Not suitable for handling severe mental health crises or suicidal ideation
• General Limitations: As with all language models, may occasionally generate inappropriate or inaccurate responses

Usage

Basic Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("Skshackster/gemma3-270m-mental-health-fine-tuned-gguf")
tokenizer = AutoTokenizer.from_pretrained("Skshackster/gemma3-270m-mental-health-fine-tuned-gguf")

# Prepare conversation
messages = [{
    "role": "user",
    "content": [{"type": "text", "text": "I've been feeling really anxious lately about work."}]
}]

# Generate response
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        temperature=1.0,
        top_p=0.95,
        top_k=64,
        do_sample=True
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Recommended Inference Settings

• Temperature: 1.0
• Top-p: 0.95
• Top-k: 64
• Max New Tokens: 64-256 (depending on desired response length)

Training Data

The model was trained on a carefully curated dataset of mental health conversations incorporating:

• CBT-based therapeutic dialogues
• Empathetic response patterns
• Crisis de-escalation techniques
• Mindfulness and coping strategies
• Educational mental health content

Data Volume: 5M+ tokens of high-quality conversational data

Evaluation and Performance

The model demonstrates strong performance in:

• Empathetic response generation
• CBT technique application
• Maintaining therapeutic conversation flow
• Appropriate boundary setting
• Educational content delivery

Ethical Considerations

Safety Measures

• Trained to redirect users to professional help when appropriate
• Designed to avoid giving specific medical advice
• Incorporates safety guidelines for mental health conversations
• Includes appropriate disclaimers about professional treatment

Bias and Fairness

• Efforts made to ensure inclusive and culturally sensitive responses
• Regular evaluation for potential biases in mental health recommendations
• Continuous monitoring for harmful or inappropriate outputs

Technical Specifications

• Architecture: Gemma-3 (Transformer-based)
• Context Length: 4000 tokens
• Precision: BF16
• Hardware Requirements: Compatible with consumer GPUs (4GB+ VRAM recommended)
• Inference Speed: Optimized for real-time conversation

Files and Formats

• Standard Model: PyTorch format compatible with Transformers library
• GGUF Format: Available for llama.cpp and Ollama integration
• Quantization: BF16 precision maintained for quality

Citation

If you use this model in your research or applications, please cite:

@misc{srivastava2025gemma3mentalhealth,
  title={Gemma-3 270M Mental Health Fine-tuned Model},
  author={Saurav Kumar Srivastava},
  year={2025},
  howpublished={\url{https://huggingface.co/Skshackster/gemma3-270m-mental-health-fine-tuned-gguf}},
}

Contact and Support

Developer: Saurav Kumar Srivastava

• For questions, issues, or collaboration inquiries, please open an issue in the model repository

Acknowledgments

• Google for the Gemma-3 base model
• Unsloth for the efficient fine-tuning framework
• Mental Health Community for supporting ethical AI development in therapeutic applications

Disclaimer

This model is designed for educational and supportive purposes only. It should not be used as a substitute for professional mental health treatment. If you are experiencing a mental health crisis, please contact a licensed mental health professional or emergency services immediately.

---