hxrikp/qwen3-4b-medical-reasoning-qlora

Qwen3-4B Medical Reasoning (QLoRA adapter)

Title: Qwen3-4B Medical Reasoning (QLoRA) Subtitle: A QLoRA adapter fine-tuning Qwen3-4B-Instruct on medical chain-of-thought reasoning, trained on a single free Kaggle T4 - the SFT checkpoint from the "Confidence Tax" study.

Reusable as both the Kaggle model card and the Medium/blog blurb. ⚠️ Research & education only - NOT for clinical or patient-facing use.

Model Summary

A LoRA adapter that fine-tunes Qwen/Qwen3-4B-Instruct-2507 on medical chain-of-thought reasoning. It is the supervised-fine-tuned ("after") checkpoint from a controlled calibration study of small medical LLMs. The headline finding: this fine-tune makes the model's internal (token-probability) confidence markedly better calibrated, while its verbalized (self-reported) confidence does not improve - the two diverge. Trained entirely on a single free Kaggle T4.

Model Characteristics

• Base: Qwen3-4B-Instruct-2507 (Apache-2.0), 4B params, text-only.
• Method: QLoRA - 4-bit NF4 (double-quant), LoRA r=16, α=32, dropout 0.05 on q/v/o/gate/up/down projections. Trainable params well under 1% of the model.
• Training: 500 steps (~1.33 epochs), seq len 512, effective batch 16, lr 2e-4, fp16. Final/avg train_loss = 1.477 (from ~1.7), grad-norm stable ~0.27, no divergence. ~3.5 h on one T4.
• Format: PEFT LoRA adapter (adapter_config.json + adapter_model.safetensors). Not a standalone model - load on top of the base. (Base tokenizer is used; not bundled.)

Data Overview

• FreedomIntelligence/medical-o1-reasoning-SFT (Apache-2.0): chain-of-thought medical Q&A, GPT-4o-generated reasoning validated by a medical verifier. No PHI. ~6,000 examples used.

Evaluation Results

Before (base) vs. after (this adapter) on MedQA + MedMCQA (0-shot, n≈800), with 95% paired bootstrap CIs (2000 resamples). Bold = the change is statistically significant (CI excludes 0).

Internal confidence (max-softmax over option logits - the deployable signal):

metric	base	+this adapter	Δ [95% CI]
Accuracy ↑	0.611	0.576	-0.036 [-0.063, -0.007]
ECE ↓ (calibration error)	0.351	0.085	-0.266 [-0.287, -0.220]
AURC ↓ (selective prediction)	0.223	0.236	+0.013 [-0.013, +0.031] (n.s.)
Abstention rate	0.125	0.086	-0.039 [-0.060, -0.019]

Verbalized confidence (the "Confidence: NN" it writes after reasoning; unparsed dropped): ECE rises 0.232 to 0.342 (+0.073 [+0.009, +0.130]) - i.e. internal calibration improves while spoken confidence does not. Trust the logits, not the self-report.

Takeaway: internal calibration error roughly quartered (no "confidence tax" in selective prediction), at a small accuracy cost. Replicated on MedGemma-4B (ECE 0.304 to 0.126). Full method, CIs, and the matched-accuracy decomposition: full method, CIs, and code: https://github.com/Mr-Neutr0n/confidence-tax-study

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-4B-Instruct-2507"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "<this-adapter-path>")  # load the LoRA adapter

License & citation

• License: Apache-2.0 (base + data are Apache-2.0).
• Code and analysis: https://github.com/Mr-Neutr0n/confidence-tax-study