ALIA-es-gemma-clinical-psychology-sft

This repository contains a supervised fine-tuned (SFT) version of the Gemma 4 E4B IT model, optimized for Spanish psychological counseling and empathetic therapeutic dialogue.

This model is the result of a Supervised Fine-Tuning (SFT) process on the google/gemma-4-E4B-it base model, using a curated multi-turn psychology dataset in Spanish containing professional therapist dialogues.

This is a pilot training run for research purposes. It is not an official release and has not been validated for general deployment.

DISCLAIMER: This model is a domain-specific proof-of-concept for therapeutic guidance and research. It has NOT been clinically validated and has not undergone regulatory review. It may produce incorrect, unsafe, or misleading psychological advice. Do not use this model as a substitute for professional therapy, diagnosis, or psychiatric treatment. Always consult a qualified psychologist or healthcare professional.

---

Model Details

Description

This model is a Transformer-based decoder-only language model that builds on the Gemma 4 E4B IT architecture through SFT alignment targeted to psychological support in Spanish.

SFT Fine-Tuning: The model was fine-tuned using Supervised Fine-Tuning (SFT) to align responses with empathetic, active listening strategies. The dataset consists of multi-turn dialogues between patients and therapists, reinforcing safe, validating, and explorative conversational practices.

Architecture

Base Model	google/gemma-4-E4B-it
Architecture	Dense + PLE (Parameter-Layer-Embedding)
Total Parameters	8,000,000,000 (8B)
Effective Parameters	4,500,000,000 (4.5B)
Layers	42
Shared KV cache layers	18
Context length	4,096 (Configured)
Attention Pattern	Alternating local sliding-window (512 tokens) and global full-context
Precision	bfloat16
Flash attention	❌ (Disabled)

Hyperparameters

Parameter	Value
Sequence length	4,096
Sample packing	false
Pad to sequence length	true
Num. epochs	20
Save steps	40
Eval steps	20
Logging steps	5
Optimizer	adamw_torch
Learning rate	5e-5
LR scheduler	cosine
Warmup ratio	0.05
Weight decay	0.01
Micro batch size	1
Gradient accumulation steps	2
Gradient checkpointing	true
Val set size	0.05
Seed	42
BF16	true
FP16	false
Fine-Tuning Method	SFT (Supervised Fine-Tuning)
Adapter	LoRA
LoRA R	16
LoRA Alpha	32
LoRA Dropout	0.05
LoRA Target Modules	`model.language_model.layers.[\d]+.(_checkpoint_wrapped_module.)?(mlp

---

Intended Use

Direct Use

The model is intended for research, development and support applications within Spanish psychological counseling and active listening contexts. Representative use cases include:

• Assisting mental health professionals with drafts or suggestions for empathetic response strategies.
• Role-playing and scenario training for psychology students.
• Analysis and study of automated fine-tuning in clinical-adjacent communication settings.

Out-of-scope Use

This model is not approved for clinical use or autonomous deployment. It must not be used as a primary source for psychological diagnosis, psychiatric treatment decisions, or crisis intervention. Any deployment that impacts patient safety requires extensive validation, risk assessment and regulatory clearance.

---

How to use

Python Example

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "SINAI/ALIA-es-gemma-clinical-psychology-sft"

# System prompt used to steer the model towards empathetic therapy
system_prompt = (
    "Eres un terapeuta psicológico empático y profesional. "
    "Escucha activamente al paciente y responde de forma apropiada, "
    "validando sus emociones y explorando su experiencia."
)

# Example conversation structure (using the Gemma 4 chat template style)
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Hola. Pues llevo desde hace mucho sintiendome con mucha ansiedad por basicamente casi todo"}
]

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Apply the chat template
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)

# Decode the generated tokens (skipping the prompt part)
generated_tokens = outputs[0][inputs.input_ids.shape[1]:]
print(tokenizer.decode(generated_tokens, skip_special_tokens=True))

---

Data

SFT Fine-Tuning Data

Data Availability: The dataset used for training is publicly available on Hugging Face at SINAI/ALIA-es-clinical-psychology-dialogues.

To adapt the model to empathetic psychological counseling in Spanish we used the following resource:

• Supervised Fine-Tuning (SFT)
  • Dataset: ALIA-es-clinical-psychology-dialogues.jsonl (containing 67 multi-turn sessions)
  • Description: A multi-turn dataset structured with a conversations history. The target responses (assistant) are human-curated/edited therapist dialogues that prioritize validation, empathy, active listening, and safe, explorative therapeutic interaction in Spanish.

---

Additional Information

License

Apache License, Version 2.0

Citation

@misc{ALIA-es-gemma-clinical-psychology-sft,
    title={ALIA-es-gemma-clinical-psychology-sft: Empathetic Psychology SFT Model for Spanish},
    author={SINAI Research Group},
    year={2026},
    publisher={HuggingFace},
    howpublished={\url{https://huggingface.co/SINAI/ALIA-es-gemma-clinical-psychology-sft}}
}

Please also cite the base model and family:

@misc{gemma4_2026,
    title={Gemma 4: Open Weights Multimodal Models},
    author={Google DeepMind},
    year={2026},
    url={https://deepmind.google/gemma}
}

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project ALIA.

---

Contact: ALIA Project - SINAI Research Group - Universidad de Jaén

More Information: SINAI Research Group | ALIA-UJA Project