sumitkalamkar/mistral-7b-medical-qa-qlora

Mistral-7B Medical QA — QLoRA Fine-tuned

Author: Sumit Pandurang Kalamkar
Platform: Google Colab (Tesla T4 16GB)
Date: March 2026

Model Description

Mistral-7B-v0.1 fine-tuned on PubMed QA (pqa_labeled) using QLoRA for biomedical question answering. Trained on 630 samples using only 1.2% of total parameters on a single free T4 GPU.

Results

Model	ROUGE-1	ROUGE-2	ROUGE-L
Mistral-7B (zero-shot)	0.2138	0.0707	0.1532
This model (QLoRA)	0.3445	0.1249	0.2288
Improvement	+61%	+76%	+49%

Training Details

Property	Value
Base Model	Mistral-7B-v0.1
Dataset	PubMed QA (pqa_labeled)
Train Samples	630
Test Samples	70
LoRA Rank	8
LoRA Alpha	16
Target Modules	q_proj, v_proj
Trainable Params	~1.2%
Quantization	4-bit NF4
Learning Rate	2e-4
Total Steps	600
Final Loss	1.043
GPU	Tesla T4 16GB
Training Time	~3.86 hours

Sample Outputs

Q: What are the symptoms of Type 2 diabetes?

The symptoms of type 2 diabetes include increased thirst, increased urination, increased hunger, weight loss, blurred vision, and fatigue. These symptoms occur because the body's cells are starved of glucose.

Q: How does insulin work?

Insulin is a hormone produced by beta cells in the pancreas. It helps cells take in glucose from the blood as fuel. When cells don't respond properly, glucose stays in the blood — this is called insulin resistance.

Q: What causes high blood pressure?

High blood pressure has many causes including narrowed arteries, kidney problems, being overweight, and excess salt in the diet.

How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained(
    "sumitkalamkar/mistral-7b-medical-qa-qlora"
)
model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(
    model,
    "sumitkalamkar/mistral-7b-medical-qa-qlora"
)

def ask(question):
    prompt = f"<s>[INST] You are an expert medical AI assistant. Question: {question} [/INST] Answer:"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=200,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id
        )
    return tokenizer.decode(
        outputs[0][inputs['input_ids'].shape[1]:],
        skip_special_tokens=True
    ).strip()

print(ask("What are the symptoms of Type 2 diabetes?"))

References

1. Dettmers et al. (2023). QLoRA. arXiv:2305.14314
2. Hu et al. (2021). LoRA. arXiv:2106.09685
3. Jin et al. (2019). PubMedQA. EMNLP 2019
4. Jiang et al. (2023). Mistral 7B. arXiv:2310.06825

Links

• GitHub: https://github.com/Sumitkalamkar/mistral-7b-medical-qa-qlora
• Research Report: See RESEARCH_REPORT.md in Files tab