Instructions to use lianghsun/gemma-3-tw-270m-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/gemma-3-tw-270m-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/gemma-3-tw-270m-it")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/gemma-3-tw-270m-it")
model = AutoModelForMultimodalLM.from_pretrained("lianghsun/gemma-3-tw-270m-it")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lianghsun/gemma-3-tw-270m-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/gemma-3-tw-270m-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/gemma-3-tw-270m-it

SGLang

How to use lianghsun/gemma-3-tw-270m-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/gemma-3-tw-270m-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/gemma-3-tw-270m-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/gemma-3-tw-270m-it with Docker Model Runner:
```
docker model run hf.co/lianghsun/gemma-3-tw-270m-it
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for gemma-3-tw-270m-it

gemma-3-tw-270m-it 是針對中華民國台灣語境設計的繁體中文輕量級指令微調模型。本模型建構於 lianghsun/gemma-3-270m-tw（Gemma-3 270M 的繁中持續預訓練版本）之上，並以繁中對話資料完成指令微調，適合在端側、邊緣裝置與瀏覽器內等資源受限的情境下提供輕量級的繁中對話能力。

⚠️ 規格重點：本模型為 270M 參數小型語言模型（SLM）、純文本單模態。

Model Details

繼 Gemma-3 系列模型推出後，270M 級別的小模型在端側與低成本部署的情境上有獨特優勢，但原版繁中能力與台灣本地語境理解相當有限。gemma-3-tw-270m-it 即為解決此差距而設計的指令微調版本：先以繁中語料持續預訓練，再以台灣常見任務之指令對話完成 SFT，使其在輕量級規模下仍保有可用的繁中互動能力。

核心特點 (Key Features)

小型化、可端側部署：270M 參數，可在筆電 CPU、邊緣裝置或瀏覽器（搭配 transformers.js／ONNX 量化版本）運行。
台灣語境對齊：訓練資料以繁體中文與台灣常見任務為主，避免一般小模型「會繁體中文卻不懂台灣」的問題。
作為下游微調起點：可作為更小型專案、領域對話模型（如 keyboard-warrior）的微調基底。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: lianghsun/gemma-3-270m-tw
Model type: Gemma3ForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: gemma (Google usage license)

Model Sources

Repository: lianghsun/gemma-3-tw-270m-it

Citation

@misc{gemma_3_tw_270m_it,
  title        = {gemma-3-tw-270m-it: A Lightweight Traditional Chinese Instruction-Tuned Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/lianghsun/gemma-3-tw-270m-it}}
}