qwen25-7b-empathy

qwen25-7b-empathy is a LoRA adapter fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit for emotionally supportive conversation. The goal is to make Qwen2.5-7B respond less like a generic daily chatbot and more like an empathetic support assistant: validating feelings first, reflecting the user's emotional state, and offering gentle next steps when appropriate.

This repository contains the PEFT adapter weights, tokenizer files, chat template, and model card. It is not a merged full-weight model.

Model Details

• Developed by: JamieYCR
• Model type: Qwen2.5-7B Instruct LoRA adapter
• Base model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
• Library: PEFT / Transformers / TRL / Unsloth
• Language: English
• Task: text generation, emotional-support conversation
• Adapter size: about 154 MB

Intended Use

This adapter is intended for research, demos, and experiments around empathetic dialogue generation. It can be used for:

• emotionally supportive chatbot responses
• empathy-focused response generation
• comparison with a base Qwen2.5 instruction model
• classroom or portfolio demonstrations of LoRA fine-tuning

It should not be used as a replacement for professional mental health care, medical advice, legal advice, or emergency support. For crisis or self-harm situations, route users to qualified human help and local emergency resources.

Training Data

The final training mix emphasized empathy-focused and harder emotional-support examples rather than ordinary small talk. The mix included:

• EmpatheticDialogues
• ESConv emotional-support conversations
• CounselChat-style counseling QA examples
• locally generated hard-case domain examples from difficult emotional scenarios
• handwritten empathy-boost examples
• safety-oriented examples

Daily small-talk data was removed from the boosted run so the model would learn deeper validation and emotional reflection instead of casual chatbot replies.

Training Procedure

The adapter was trained with supervised fine-tuning using QLoRA/LoRA on Qwen2.5-7B Instruct.

Key settings from the final boosted run:

• LoRA rank: 16
• LoRA alpha: 16
• LoRA dropout: 0.0
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Training epochs: 3
• Learning rate: 7.5e-5
• PEFT version: 0.19.1
• Final local adapter path: outputs/qwen25-7b-empathy-boosted/final_lora

Example Behavior

Prompt:

Emotion: sadness 0.62, disappointment 0.28
Message: I keep telling everyone I'm just tired, but honestly I don't remember the last time I felt excited for anything.

Example response:

That sounds really painful, like the absence of hope feels harsh and physical. It makes sense that you feel discouraged; you deserve support around how hard this has been, not just pressure to feel better quickly.

How to Load

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
import torch

adapter_id = "JamieYCR/qwen25-7b-empathy"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoPeftModelForCausalLM.from_pretrained(
    adapter_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

messages = [
    {
        "role": "system",
        "content": "You are a deeply empathetic emotional-support assistant. Validate feelings first, reflect the user's emotional state, and avoid generic small talk.",
    },
    {
        "role": "user",
        "content": "I keep telling everyone I'm just tired, but honestly I don't remember the last time I felt excited for anything.",
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True,
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=220,
    temperature=0.7,
    top_p=0.9,
)

print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Limitations and Risks

• The model can sound emotionally supportive, but it does not understand a user's full real-world context.
• It may over-validate, miss clinical risk, or give advice that is too general.
• It may produce unsafe or incomplete answers in crisis situations.
• It inherits limitations and biases from the base model and training data.
• It should be evaluated carefully before use in any user-facing support setting.

Recommended Safeguards

• Add crisis detection and human escalation for self-harm, abuse, violence, or emergency situations.
• Make it clear to users that the system is not a therapist or medical professional.
• Test responses across different emotional states and user backgrounds.
• Monitor for generic, dismissive, overly directive, or unsafe responses.

Citation

If you use this adapter, cite the base Qwen2.5 model and the datasets used in your training/evaluation notes where applicable.

Contact

For questions about this fine-tuned adapter, contact the repository owner on Hugging Face: JamieYCR.