---
language:
- th
library_name: transformers
pipeline_tag: text-generation
tags:
- medical
- rag
- citation
- question-answering
- thai
---

# ThaiLLM-30B-IQ (Medical)

## Model Description

**ThaiLLM-30B-IQ (Medical)** is a Thai-language large language model specialized for **medical information query** and **citation-grounded question answering**.

The model is fine-tuned from **ThaiLLM-30B** using supervised fine-tuning (SFT) with medical-domain data. It is designed for **retrieval-augmented generation (RAG)** workflows, where answers must be generated **only from provided medical contexts** and returned with explicit citations.

> This model does not include a retriever and must be used with externally supplied medical documents.

---

## Intended Use

### Suitable for
- Medical RAG systems (Thai)
- Medical document question answering
- Citation-grounded medical QA
- Medical education and evaluation

### Not suitable for
- Medical diagnosis or treatment decisions
- Patient-facing clinical advice
- Emergency or critical-care use

---

## Model Details

- **Base model:** ThaiLLM-30B
- **Domain:** Medical  
- **Model type:** Decoder-only causal language model  
- **Fine-tuning:** Supervised fine-tuning (LoRA)  
- **Language:** Thai  
- **Precision:** bfloat16  

---

## Performance (Medical Information Query Setting)

| Model | Response (BLEU) | Citations (Jaccard) |
|------|-----------------|---------------------|
| ThaiLLM-30B | 0.406 | 0.1786 |
| **ThaiLLM-8B-SFT-IQ (Medical)** | **0.4331** | **0.5458** |

---

## Training

- **Framework:** LLaMA-Factory  
- **Method:** LoRA-based supervised fine-tuning  

| Hyperparameter | Value |
|---------------|-------|
| Learning rate | 2e-4 |
| LoRA rank | 16 |
| LoRA alpha | 16 |
| Sequence length | 4096 |
| Epochs | 3 |
| Batch size | 4 |

Training data consists of Thai medical question–answer pairs with context grounding and citation supervision.
- **Training Script**
- ```bash
  llamafactory-cli train \
    --stage sft \
    --do_train True \
    --model_name_or_path ${MODEL_PATH} \
    --preprocessing_num_workers 16 \
    --deepspeed ./examples/deepspeed/ds_z2_config.json \
    --finetuning_type lora \
    --template qwen3 \
    --flash_attn auto \
    --dataset thaillm-SFT \
    --cutoff_len 4096 \
    --learning_rate 2e-4 \
    --num_train_epochs 3.0 \
    --max_samples 100000 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 4 \
    --gradient_checkpointing True \
    --gradient_checkpointing_kwargs '{"use_reentrant":false}' \
    --lr_scheduler_type cosine \
    --max_grad_norm 1.0 \
    --logging_steps 5 \
    --save_steps 100 \
    --warmup_steps 0 \
    --packing False \
    --enable_thinking True \
    --report_to tensorboard \
    --quantization_bit 4 \
    --output_dir "./checkpoints/ThaiLLM-30B-SFT-IQ" \
    --overwrite_output_dir True \
    --bf16 True \
    --plot_loss True \
    --trust_remote_code True \
    --include_num_input_tokens_seen True \
    --optim adamw_8bit \
    --lora_rank 16 \
    --lora_alpha 16 \
    --upcast_layernorm True \
    --ddp_timeout 180000000 \
    --lora_dropout 0 \
    --lora_target all \
    --freeze_vision_tower True \
    --freeze_multi_modal_projector True \
    --use_unsloth False \
    --seed 1234
  ```

---

## Usage

The model is optimized for **strict JSON output** and **context-only medical answers**.

### Expected Instruction Format

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ThaiLLM-30B-SFT-IQ"

tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

system_prompt = """\
Answer in JSON format with citations only.
Use only the provided medical contexts to answer the question.
Include the fact IDs that support your answer in the following format:
{"answer": "<ANSWER_TEXT>", "citations": ["<FACT_ID1>", "<FACT_ID2>"]}

If unknown, respond with:
{"answer": "unknown", "citations": []}
"""

question = "การบาดเจ็บจากอุบัติเหตุที่ข้อต่อขากรรไกรสามารถทำให้อาการปวดร้าวไปที่ใดเมื่อเคี้ยวอาหาร?"

facts = """\
[1] การบาดเจ็บจากอุบัติเหตุที่ข้อต่อขากรรไกรสามารถทำให้ปวดร้าวไปที่หูเมื่อเคี้ยวอาหาร
[2] ข้อต่อขากรรไกรอักเสบ (TMJ) สามารถทำให้ปวดบริเวณใกล้ติ่งหูและร้าวไปที่หูเมื่อเคี้ยวอาหาร
"""

prompt = f"""{system_prompt}

Question:
{question}

Facts:
{facts}
"""

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        inputs,
        max_new_tokens=256,
        do_sample=False,
        temperature=0.0,
    )

generated = outputs[0, inputs.shape[-1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))
```