CoRBs/telemed-k-gemma4-e4b-ro-medical

TeleMed Hearth — Gemma 4 E4B Romanian Rural Triage Adapter

A LoRA adapter fine-tuned on top of unsloth/gemma-4-E4B-it-unsloth-bnb-4bit for Romanian rural-elderly medical triage. Co-developed with practicing physicians at Clinica Medicală Dr. Bogheanu (Brănești, Dâmbovița County, Romania).

**Submitted to the Kaggle Gemma 4 Good Hackathon, May 2026.

Deployment compatibility note

The TeleMed Hearth Android application — the parent project this adapter was trained for — ships the unmodified base gemma-4-E4B-it.litertlm on device, driven by the same engineered system prompt that conditioned the adapter training. The reason is technical: LiteRT-LM 0.11.0 does not currently support merging a PEFT LoRA adapter into the .litertlm runtime format, and MediaPipe's available converters target only GEMMA_2B, GEMMA2_2B, and PHI_2 — there is no public conversion path from a Gemma 4 LoRA adapter to a deployable on-device artifact at this time.

This adapter is therefore published for:

• Server-mediated deployments of Romanian-language medical triage where PEFT-adapter loading is supported (transformers, vLLM, TGI, Unsloth runtime).
• Researchers and clinics evaluating Romanian medical-register adaptation on Gemma 4 E4B.

The held-out evaluation below shows what the adapter actually buys over the base model with the same system prompt, and what it does not.

Intended use

Performs structured triage in Romanian for elderly rural patients before a family-doctor consultation. Each assistant turn emits a stringified JSON object matching this schema:

{
  "response": "<Romanian text shown to the patient>",
  "emergency": false,
  "confidence": 0.9,
  "priority": "normal",
  "ready_to_finalize": false,
  "category": "duration"
}

Field domains:

• priority: "normal" | "urgent" | "emergency"
• category: "duration" | "intensity" | "associated_symptoms" | "context" | "history" | "close" | "emergency"

For vital emergencies (chest pain + dyspnea, stroke, severe bleeding, anaphylaxis, loss of consciousness) the adapter emits the canonical phrase "Sunați 112 imediat." with emergency: true. For suicidal ideation it includes the Romanian Anti-Suicide Hotline (0800 801 200, Alianța Română de Prevenție a Suicidului).

The model is conditioned to:

• Ask one clarifying question per turn
• Use formal Romanian address (dumneavoastră → dvs.)
• Cap sentences at 30 words
• Never diagnose, never name medications, never interpret tests

Training data

121 synthetic Romanian rural-elderly triage dialogues co-developed with two practicing physicians (a consultant pediatrician and a consultant family medicine specialist) at a partner rural clinic in Dâmbovița County, Romania. The dialogues cover 13 themes:

• Chronic disease management — hypertension, type 2 diabetes, COPD, heart failure, arthritis
• General symptoms — cough, fever, fatigue, dizziness, headache, back pain
• Dermatology, gastrointestinal, mental health, urinary, vision/hearing
• Medication management confusion (boundary case — the adapter must not name drugs)
• Emergencies — chest pain + dyspnea, stroke, anaphylaxis, severe bleeding, loss of consciousness, suicidal ideation, thunderclap headache

Split: 109 training dialogues / 12 stratified held-out evaluation dialogues. Zero overlap. seed=42. Each row is prepended with the production system message so training and inference conditioning match byte-for-byte. The eval set covers 9 themes and 3 emergency patterns.

Training config

Parameter	Value
Base model	unsloth/gemma-4-E4B-it-unsloth-bnb-4bit
Method	QLoRA (4-bit NF4 + BF16 compute)
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Epochs	3
Effective batch size	4 (per_device=1, grad_accum=4)
Learning rate	2e-4 (cosine schedule, warmup 0.1)
Optimizer	adamw_8bit
Weight decay	0.01
Max grad norm	0.3
BF16	true
Seed	42
Trainable params	42.4M (0.67% of 6.34B total)
Final train loss	0.6100
Mid-training eval loss	2.184
Hardware	NVIDIA GB10 (Grace Blackwell, ARM64, 128 GB unified memory)
Framework	Unsloth 2026.5.2 + transformers 5.5.0 + trl 0.24.0 + peft 0.19.1

Evaluation — head-to-head against base + system prompt

12 stratified held-out dialogues, greedy decoding, max_new_tokens=256. Same system prompt applied to both the adapter and the unmodified base model.

Metric	Adapter	Base + system prompt
JSON parse rate	12/12	12/12
Schema compliance (6 fields)	12/12	12/12
Emergency true positives	3/3	3/3
Emergency false negatives	0	0
Emergency false positives	0	0
Medication blocklist hits	0 (target: 0)	0 (target: 0)
Greeting-rule violations	0	3
Canonical-phrase fidelity (verbatim "Sunați 112 imediat.")	0/3 (paraphrases)	2/3
Avg response length	18.2 words	22.7 words

Reading the numbers. Both systems achieve perfect safety routing — every emergency case in the held-out set is correctly flagged with emergency: true. Where they diverge:

• The adapter wins on register and discipline. Zero greeting-rule violations vs. 3 for the base; shorter, more conversationally appropriate responses (18.2 vs. 22.7 words average).
• The base wins on canonical-phrase fidelity. On 2 of 3 emergency cases the base reproduces "Sunați 112 imediat." verbatim; the adapter paraphrases (e.g., "Vă rog să sunați imediat 112. Aceasta este o urgență medicală."). The paraphrase is still routed correctly by the application layer (which routes on the emergency: true flag, not the response string), but for systems that match on the canonical phrase directly, fidelity matters.
• On the anaphylaxis case (synth-116), the adapter stayed anchored to the patient's stated symptoms; the base drifted toward off-symptom chest-pain clarification.

Engineering takeaway. 121 synthetic dialogues teach Gemma 4 E4B conversational discipline reliably, but at this dataset size the base's pre-training priors on canonical safety phrases dominate. For on-device deployment where the application routes on the emergency boolean and not the response text, the base model with the engineered system prompt is the more practical choice — equal safety routing, no PEFT-merge dependency.

Full per-dialogue outputs and metrics: eval_outputs.jsonl, eval_report.md, eval_metrics.json in this repo (plus the corresponding eval_base_* files for the head-to-head comparator).

Usage

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="CoRBs/telemed-k-gemma4-e4b-ro-medical",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

# Sampling parameters used in evaluation: greedy decoding (temperature=0).
# Production conditioning in TeleMed Hearth: temp=0.3, top_p=0.9, top_k=40.
# Use the same Romanian system prompt the adapter was trained with — see
# ai_engine_service.dart in the parent TeleMed Hearth Flutter app for the
# exact production text.

Limitations & responsible use

• Not a diagnostic tool. Produces a structured triage report for a human family physician to review and finalize. Never deploy as the sole decision-maker.
• Synthetic training data, single-clinic review. All 121 dialogues were co-developed and reviewed by two practicing physicians at one rural Romanian site. Broader clinical validation is required before any scaled deployment.
• Romanian rural-elderly register only. Performance on other dialects, age groups, or non-medical conversations is undefined.
• Confidence value drift. The adapter emits confidence: 0.9 for nearly all outputs. The application layer routes on the emergency flag, not on confidence. Do not use confidence as a clinical signal.
• Canonical-phrase paraphrasing. As noted in the evaluation, the adapter paraphrases canonical emergency phrases on some cases. Systems that need exact-phrase routing should match on the emergency boolean instead of substring-matching the response text.
• No vision encoder fine-tune. The training run merged adapter weights into text projection modules only; multimodal photo inputs at inference time use the unmodified vision encoder of the base.

Author

[Rareș Bogheanu] — adapter author. Senior QA Lead, 20 years, now building and auditing production AI agent systems. Available for AI agent reliability and local-first prototype work at · LinkedIn · GitHub.

Citation

@misc{telemed_hearth_2026,
  title  = {TeleMed Hearth — Romanian Rural-Elderly Medical Triage Adapter for Gemma 4 E4B},
  author = {Bogheanu, Rareș and collaborators},
  year   = {2026},
  url    = {https://huggingface.co/CoRBs/telemed-k-gemma4-e4b-ro-medical}
}

License

Apache 2.0 (inherits from base model). Training data: synthetic, co-developed with the physicians at Clinica Medicală Dr. Bogheanu.