---
base_model: google/medgemma-27b-text-it
library_name: peft
license: apache-2.0
tags:
  - medgemma
  - medical
  - nhs
  - clinical-documentation
  - lora
  - qlora
  - kaggle-competition
---

# Clarke LoRA Adapter - MedGemma 27B for NHS Clinic Letters

QLoRA fine-tuned adapter for `google/medgemma-27b-text-it`.

Built for the [MedGemma Impact Challenge](https://www.kaggle.com/competitions/medgemma-impact-challenge) on Kaggle. Part of the [Clarke](https://huggingface.co/spaces/yashvshetty/clarke) clinical documentation system.

## Training Details

- **Method**: QLoRA (4-bit NF4, LoRA rank 16, alpha 32, dropout 0.05)
- **Target modules**: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (attention + MLP)
- **Training data**: 5 gold-standard NHS clinic letters (separate Assessment, Plan, and Advice to Patient sections)
- **Epochs**: 3
- **Learning rate**: 2e-4, 8-bit AdamW optimiser
- **Hardware**: NVIDIA A100 80 GB (HuggingFace Spaces)
- **Training loss**: 2.09 to 1.30 (38% reduction)
- **Competition**: [MedGemma Impact Challenge](https://www.kaggle.com/competitions/medgemma-impact-challenge)

## Evaluation

The adapter was evaluated against FHIR-aligned gold-standard NHS clinic letters across five patients. The base model with optimised prompting outperformed the adapter at this data scale (n=5):

| Metric | Base Model | LoRA Adapter |
|--------|-----------|-------------|
| BLEU-1 | 0.82 | 0.79 |
| BLEU-4 | 0.61 | 0.39 |
| ROUGE-L | 0.74 | 0.60 |

This result is consistent with the literature on fine-tuning large language models with very small datasets. The adapter demonstrates the fine-tuning pipeline and would benefit from a larger clinical corpus (50-200+ examples). Full evaluation methodology is in the [Clarke evaluation report](https://huggingface.co/spaces/yashvshetty/clarke/blob/main/evaluation/EVALUATION.md).

## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("google/medgemma-27b-text-it")
model = PeftModel.from_pretrained(base_model, "yashvshetty/clarke-medgemma-27b-lora")
```

## Links

- [Clarke Live Demo](https://yashvshetty-clarke.hf.space)
- [Clarke Source Code](https://github.com/yvs-tinker/clarke)
- [Evaluation Report](https://huggingface.co/spaces/yashvshetty/clarke/blob/main/evaluation/EVALUATION.md)