---
license: other
license_name: health-ai-developer-foundations
license_link: https://developers.google.com/health-ai-developer-foundations/terms
base_model: google/medgemma-1.5-4b-it
library_name: peft
tags:
- medical
- healthcare
- lora
- qlora
- peft
- calibration
- selective-prediction
---

# MedGemma-1.5-4B Medical Reasoning (QLoRA adapter)

**Title:** MedGemma-4B Medical Reasoning (QLoRA)
**Subtitle:** A QLoRA adapter fine-tuning MedGemma-1.5-4B-it on medical chain-of-thought reasoning, trained on a single free Kaggle T4 - the cross-model replication checkpoint from a small-LLM calibration study.

> Reusable as both the Kaggle **model card** and the Medium/blog blurb.
> ⚠️ Research & education only - **NOT for clinical or patient-facing use.**

## Model Summary
A **LoRA adapter** that fine-tunes [`google/medgemma-1.5-4b-it`](https://huggingface.co/google/medgemma-1.5-4b-it)
on **medical chain-of-thought reasoning**. It is the supervised-fine-tuned ("after") checkpoint that
**replicates**, on a medical-specialist base, the central finding of the companion Qwen3-4B study: this
fine-tune makes the model's **internal** (token-probability) confidence markedly **better calibrated**
and dramatically **less over-cautious**. Trained entirely on a **single free Kaggle T4**.

## Model Characteristics
- **Base:** MedGemma-1.5-4B-it (HAI-DEF license, **gated**), 4B params, multimodal (image+text); used
 here as a text causal-LM.
- **Method:** QLoRA - 4-bit NF4 (double-quant), LoRA **r=16, α=32, dropout 0.05** on
 `q/v/o/gate/up/down` projections. Trainable params well under 1% of the model.
- **Training:** ~250 steps (past the loss plateau), seq len 1024, effective batch 16, lr 2e-4,
 **fp32 compute** (T4 doesn't support bf16; fp16 is numerically unstable for Gemma). `train_loss` 1.72 to ~1.56;
 ~4.6 h on one T4 (fp32 is ~2.7× slower per step than fp16).
- **Format:** PEFT LoRA adapter (`adapter_config.json` + `adapter_model.safetensors`). Load on top of the
 gated base (you must accept MedGemma's terms and supply an HF token).

## Data Overview
- [`FreedomIntelligence/medical-o1-reasoning-SFT`](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT)
 (Apache-2.0): chain-of-thought medical Q&A, GPT-4o-generated reasoning validated by a medical verifier.
 **No PHI.**

## Evaluation Results
Before (base) vs. after (this adapter) on **MedQA + MedMCQA** (0-shot, n≈800), **internal confidence**
(max-softmax over option logits), with **95% paired bootstrap CIs**. **Bold** = significant (CI excludes 0).

| metric | base | +this adapter | Δ [95% CI] |
|---|---|---|---|
| Accuracy ↑ | 0.556 | 0.505 | -0.051 [-0.105, +0.006] (n.s.) |
| ECE ↓ (calibration error) | 0.304 | **0.126** | **-0.178 [-0.235, -0.122]** |
| AURC ↓ (selective prediction) | 0.352 | 0.345 | -0.007 [-0.078, +0.063] (n.s.) |
| **Abstention rate** | 0.696 | **0.096** | **-0.600 [-0.634, -0.566]** |

**Takeaway:** the base model abstained on **~70%** of questions (uselessly cautious - it picks the
"defer to a clinician" option by default); the fine-tune cuts that to **~10%** *while* halving calibration
error. Accuracy change is within noise. This replicates the internal-calibration gain seen on Qwen3-4B
(ECE 0.351 to 0.085). MedGemma's *verbalized*-confidence pass is omitted (its multimodal head made
generation-time parsing unreliable), so the internal-vs-verbalized divergence is demonstrated on Qwen.
Full method + CIs: repo `healthcare/exp-004-confidence-tax/`. code at https://github.com/Mr-Neutr0n/confidence-tax-study

## Usage
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "google/medgemma-1.5-4b-it"  # gated: accept terms + pass token=...
tok = AutoTokenizer.from_pretrained(base, token=HF_TOKEN)
model = AutoModelForCausalLM.from_pretrained(
  base, torch_dtype=torch.float32, attn_implementation="eager", # fp32 + eager on T4
  device_map="auto", token=HF_TOKEN)
model = PeftModel.from_pretrained(model, "<this-adapter-path>")  # load the LoRA adapter
```

## License & citation
- **Adapter license:** Apache-2.0 (training data is Apache-2.0). **Base model:** MedGemma is governed by
 the Health AI Developer Foundations (HAI-DEF) terms - you must accept them to use the base weights.
- Code and analysis: https://github.com/Mr-Neutr0n/confidence-tax-study