Instructions to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="alpha-ai/Medical-Diagnosis-COT-Gemma3-270M")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("alpha-ai/Medical-Diagnosis-COT-Gemma3-270M")
model = AutoModelForCausalLM.from_pretrained("alpha-ai/Medical-Diagnosis-COT-Gemma3-270M")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="alpha-ai/Medical-Diagnosis-COT-Gemma3-270M",
	filename="Medical-Diagnosis-COT-Gemma3-270M.BF16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
# Run inference directly in the terminal:
llama cli -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
# Run inference directly in the terminal:
llama cli -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
# Run inference directly in the terminal:
./llama-cli -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

Use Docker

docker model run hf.co/alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

LM Studio
Jan

vLLM

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

SGLang

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alpha-ai/Medical-Diagnosis-COT-Gemma3-270M",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with Ollama:
```
ollama run hf.co/alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
```

Unsloth Studio

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alpha-ai/Medical-Diagnosis-COT-Gemma3-270M to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for alpha-ai/Medical-Diagnosis-COT-Gemma3-270M to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for alpha-ai/Medical-Diagnosis-COT-Gemma3-270M to start chatting

Atomic Chat new
Docker Model Runner
How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with Docker Model Runner:
```
docker model run hf.co/alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16
```

Lemonade

How to use alpha-ai/Medical-Diagnosis-COT-Gemma3-270M with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull alpha-ai/Medical-Diagnosis-COT-Gemma3-270M:BF16

Run and chat with the model

lemonade run user.Medical-Diagnosis-COT-Gemma3-270M-BF16

List all available models

lemonade list

Medical-Diagnosis-COT-Gemma3-270M

Alpha AI (www.alphaai.biz) fine-tuned Gemma-3 270M for medical question answering with explicit chain-of-thought (CoT). The model emits reasoning inside <think> ... </think> followed by a final answer, making it well-suited for research on verifiable medical reasoning and for internal tooling where transparent intermediate steps are desired.

⚠️Not for clinical use. This model is a research system, not a medical device. Do not use it for diagnosis or treatment decisions.

TL;DR

Base: google/gemma-3-270m-it (Gemma 3, 270M params). Access requires accepting Google’s Gemma license on Hugging Face.
Data: FreedomIntelligence/medical-o1-reasoning-SFT (en split; columns: Question, Complex_CoT, Response).
Training: SFT with LoRA via Unsloth; assistant-only loss; sequences templated in Gemma-3 chat format.
Objective: Produce <think>…</think> (reasoning) + final answer.

Intended Use & Limitations

Intended use

Research on medical reasoning, CoT interpretability, prompt engineering, dataset curation.
Internal assistants that require visible reasoning traces (to be reviewed by humans).

Out of scope / limitations

Not a substitute for clinician judgment; may hallucinate facts.
No guarantee of guideline adherence (e.g., UpToDate/NICE/ACOG).
Biases from synthetic/derived training data will propagate.

Dataset license is Apache-2.0; the base model is covered by Google's Gemma license, review these before use.

Model Details

Model family: Gemma 3 (270M).
Context window: Gemma 3 supports up to 128K tokens (practical context used during SFT was lower due to GPU limits).
Architecture: decoder-only causal LM (Gemma family).
Fine-tuning: Parameter-Efficient Fine-Tuning (LoRA) using Unsloth.

Data

Source: FreedomIntelligence/medical-o1-reasoning-SFT + Human annotated and filtered medical diagnosis data

English subset (en): ~19.7k rows with fields Question, Complex_CoT, Response. Total dataset ≈ 90,120 rows across splits/languages.
Built for advanced medical reasoning; constructed with GPT-4o and a verifier on verifiable medical problems. See dataset card and paper.
Human annotated data ~ 10k rows with similar fields.

Preprocessing

Each row rendered to a chat conversation under the Gemma-3 template.
Assistant output concatenates <think>{Complex_CoT}</think>\n{Response}.
Training objective masked to assistant tokens only (instruction masking).

Prompt & Output Format

Training prompt style (system + user + assistant):

System:
Below is an instruction that describes a task,
paired with an input that provides further context.

Write a response that appropriately completes the request.

Before answering, think carefully about the question and create a step-by-step
chain of thoughts to ensure a logical and accurate response.

### Instruction:

You are a medical expert with advanced knowledge in clinical reasoning,
diagnostics, and treatment planning.

Please answer the following medical question.

User:
### Question:

{question}

### Response:

Assistant:
<think>
{complex_cot}
</think>
{final_answer}

Generation tip (serve-time): If you don’t want to expose CoT to end-users, post-process generated text to strip the <think> ... </think> block and only show the final answer.

Python snippet to strip CoT:

import re
def strip_think(text: str) -> str:
    return re.sub(r"<think>.*?</think>\s*", "", text, flags=re.S|re.I).strip()

Quick Start

Transformers (merged weights)

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "alphaaico/Medical-Diagnosis-COT-Gemma3-270M"
tok  = AutoTokenizer.from_pretrained(repo)
mdl  = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")

prompt = """### Question:

A 65-year-old with exertional dyspnea and orthopnea presents with bilateral
pitting edema and raised JVP. What initial pharmacologic therapy is indicated?

### Response:
"""

from transformers import TextStreamer
streamer = TextStreamer(tok, skip_prompt=True, skip_special_tokens=True)

inputs = tok.apply_chat_template(
    [{"role":"system","content":"You are a careful medical reasoner."},
     {"role":"user","content":prompt}],
    tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(mdl.device)

out = mdl.generate(**inputs, max_new_tokens=256, temperature=1.0, top_p=0.95, top_k=64, streamer=streamer)

PEFT LoRA (if this repo hosts LoRA adapters)

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = "google/gemma-3-270m-it"   # requires accepting Gemma license on Hugging Face
repo = "alphaaico/Medical-Diagnosis-COT-Gemma3-270M"

tok = AutoTokenizer.from_pretrained(base)
base_mdl = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
mdl = PeftModel.from_pretrained(base_mdl, repo)  # merges LoRA at runtime
mdl.eval()

If you see a 403/404 on the base model, make sure you’ve accepted Google’s Gemma license in your Hugging Face account.

Training Procedure

Trainer: TRL SFTTrainer (supervised finetuning).
Library: Unsloth for fast loading + LoRA. Example reference repo for Gemma-3-270M is available on Hugging Face.

Key hyperparameters (demo run that produced loss ≈ 3.3):

Steps: max_steps=500 , ideall go beyond 1500 for better results.
Optimizer: adamw_8bit, weight_decay=0.01, lr_scheduler_type=linear
LR: 5e-5 (for longer runs, 2e-5 is often more stable)
Batch: per_device_train_batch_size=8, gradient_accumulation_steps=1
Warmup: warmup_steps=5
Seed: 3407
Loss masking: assistant-only (user/system tokens ignored)

Note: For very long CoT sequences on small GPUs (e.g., T4 16 GB), consider per_device_train_batch_size=1 + gradient accumulation and a larger max_seq_length. Gemma 3 supports up to 128K context, but practical fine-tuning length depends on memory. For smaller queries, everything fits well for this to be used on edge devices.

Evaluation

Training loss: ~3.3 at step 500 (demo run). Further decreased to ~2.3 with parameter tuning.
Format compliance: qualitatively verified to produce <think>…</think> followed by an answer.
No formal clinical benchmarks reported yet. Contributions welcome via the Discussions tab.

Safety, Ethics, and Responsible Use

Do not rely on outputs for patient care. Validate with authoritative sources and licensed professionals.
Be mindful of dataset artifacts and synthetic reasoning patterns.
Consider stripping CoT in user-facing apps to avoid over-trust in intermediate narratives.

License & Attribution

Base model: Google Gemma 3 — access controlled under Google’s Gemma license on Hugging Face.
Dataset: FreedomIntelligence/medical-o1-reasoning-SFT — Apache-2.0.
This fine-tune: Derivative work under the base model’s license terms. Review and comply with both licenses before distribution or commercial use.

Please cite the dataset if you use this model:

@misc{chen2024huatuogpto1medicalcomplexreasoning,
  title={HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs},
  author={Junying Chen and Zhenyang Cai and Ke Ji and Xidong Wang and Wanlong Liu and Rongsheng Wang and Jianye Hou and Benyou Wang},
  year={2024},
  eprint={2412.18925},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2412.18925}
}

Acknowledgements

Google for releasing Gemma 3.
FreedomIntelligence for medical-o1-reasoning-SFT.
Unsloth for streamlined fine-tuning utilities for Gemma 3.

Contact

Questions, issues, or contributions: open a Discussion on this repo. For enterprise collaboration with Alpha AI, reach out via the organization profile on Hugging Face or find us on www.alphaai.biz.

Appendix: Inference without CoT (server-side filter)

from transformers import AutoModelForCausalLM, AutoTokenizer
import re

repo = "alphaaico/Medical-Diagnosis-COT-Gemma3-270M"
tok  = AutoTokenizer.from_pretrained(repo)
mdl  = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")

def strip_think(txt): return re.sub(r"<think>.*?</think>\s*", "", txt, flags=re.S|re.I).strip()

def ask(question):
    user = f"### Question:\n\n{question}\n\n### Response:"
    msgs = [
      {"role":"system","content":"You are a medical expert. Think step-by-step, then answer succinctly."},
      {"role":"user","content":user},
    ]
    prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
    out = mdl.generate(**tok(prompt, return_tensors="pt").to(mdl.device), max_new_tokens=512, temperature=1.0, top_p=0.95, top_k=64)
    text = tok.decode(out[0], skip_special_tokens=True)
    return strip_think(text)

print(ask("Elderly patient with new ankle swelling on thiazolidinedione—likely cause?"))