Instructions to use Nanthasit/sakthai-context-1.5b-merged-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-1.5b-merged-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-1.5b-merged-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged-v2")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-1.5b-merged-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nanthasit/sakthai-context-1.5b-merged-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-1.5b-merged-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged-v2

SGLang

How to use Nanthasit/sakthai-context-1.5b-merged-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-1.5b-merged-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-1.5b-merged-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nanthasit/sakthai-context-1.5b-merged-v2 with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged-v2
```

sakthai-context-1.5b-merged-v2 / README.md

Nanthasit

fix: refresh family table download counts (2026-08-01T10:23Z)

d281735 verified 1 day ago

preview code

Raw

History Blame Contribute Delete

15 kB

metadata

license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
language: en
base_model: Qwen/Qwen2.5-1.5B-Instruct
datasets:
  - Nanthasit/sakthai-combined-v6
  - Nanthasit/sakthai-combined-v7
  - Nanthasit/sakthai-irrelevance-supplement
tags:
  - qwen2.5
  - qwen2
  - sakthai
  - house-of-sak
  - tool-calling
  - function-calling
  - agent
  - merged
  - cpu-inference
  - llama.cpp
  - ollama
  - conversational
  - text-generation
  - eval-results
  - benchmark
  - en
model-index:
  - name: SakThai Context 1.5B Merged V2
    results:
      - task:
          type: text-generation
        dataset:
          name: SakThai Bench v2
          type: sakthai-bench-v2
        metrics:
          - name: Selection Accuracy
            type: selection
            value: 34.9
            verified: true
            date: 2026-07-31T00:00:00.000Z
          - name: Arguments Accuracy
            type: arguments
            value: 44.2
            verified: true
            date: 2026-07-31T00:00:00.000Z
          - name: Strict Accuracy
            type: strict
            value: 34.2
            verified: true
            date: 2026-07-31T00:00:00.000Z
      - task:
          type: text-generation
        dataset:
          name: Internal tool-call smoke test
          type: custom
        metrics:
          - name: multi-tool correctness
            type: pass@3
            value: 100
            verified: true
            date: 2026-07-31T00:00:00.000Z
          - name: valid JSON rate
            type: ratio
            value: 100
            verified: true
            date: 2026-07-31T00:00:00.000Z

SakThai Context 1.5B — Merged V2

Mid-weight merged full model · Qwen2.5-1.5B · GGUF + safetensors
Best when you need more reliability than 0.5B, but still want CPU/edge inference.

Model Description

SakThai Context 1.5B Merged V2 is a merged full-weight checkpoint of Qwen2.5-1.5B-Instruct, fine-tuned for structured tool-calling and function-calling. It fills the mid-weight slot between the edge-focused 0.5B and the high-capability 7B. Use it when you need stronger argument filling and multi-turn tool behavior on a laptop/desktop with 4–8 GB RAM.

What makes it special:

🧠 1.5B parameters — stronger reasoning than 0.5B, smaller than 7B.
🗳️ Trained for structured <tool> / <tool_call> output.
📦 GGUF Q4_K_M + BF16 safetensors included.
✅ Benchmarks available: Bench v2 selection 34.9%, arguments 44.2%, strict 34.2%; internal smoke test 100% multi-tool correctness / valid JSON.
🔁 Rebuilt merged weights, ready for CPU inference with llama.cpp or transformers.

Requirements

Use these tested versions to avoid inference issues on CPU/edge hardware:

torch>=2.2
transformers>=4.45
sentencepiece>=0.2
protobuf>=3.20
accelerate>=0.27

For GGUF inference:

llama-cpp-python>=0.2.80

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ollama

The model is not published to the Ollama library, so import the local GGUF instead:

ollama create sakthai:1.5b-v2 -f Modelfile

Modelfile:

FROM ./sakthai-1.5b-q4_k_m.gguf

GGUF (llama.cpp)

# Option A — huggingface-cli
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
# Option B — direct wget
wget https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2/resolve/main/sakthai-1.5b-q4_k_m.gguf

llama-cli -m sakthai-1.5b-q4_k_m.gguf \
  --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256

Hugging Face Inference Providers (free-tier available)

Use the hosted Inference API with huggingface_hub.InferenceClient:

from huggingface_hub import InferenceClient

client = InferenceClient(model="Nanthasit/sakthai-context-1.5b-merged-v2")
messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
response = client.chat_completion(messages=messages, max_tokens=256)
print(response.choices[0].message.content)

Tool-Calling Format

The model is fine-tuned for tool calling and expects a <tools> XML block in the system prompt — the block is required: without it, the model may answer conversationally instead of emitting a tool call.

Verified output format

System: <tools>
[
  {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}
]
</tools>
User: What's the weather in Bangkok?
Assistant: <tool>get_weather</tool>

For calls with arguments, the bundled chat_template.jinja renders the standard Qwen <tool_call> JSON form:

<tool_call>
{"name": "get_weather", "arguments": {"location": "Bangkok"}}

Architecture

Property	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Parameters	1.5B (1,500,000,000)
Hidden size	1,536
Layers	28
Attention heads	12 (grouped-query, 2 KV heads)
Intermediate size	8,960
Context window	32,768 tokens
Vocab size	151,936
Precision	BF16 (safetensors) / GGUF Q4_K_M
RoPE theta	1,000,000

Training Details

Detail	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Method	SFT → merged to full weights
Training data	sakthai-combined-v6/v7 + irrelevance-supplement
Hardware	Free T4 GPU (Kaggle / Colab)

Evaluation

Setting	Selection	Arguments	Strict	Valid JSON	Multi-tool
SakThai Bench v2	34.9%	44.2%	34.2%	—	—
Internal smoke test	—	—	—	100%	100%

Notes:

Benchmarks are internal and should not be treated as independently verified.
model-index uses verified: true only because the corresponding .eval_results/ files are present in the repo.

Deployment

CPU-only

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float32,
    device_map="cpu",
    low_cpu_mem_usage=True,
)
model.eval()
messages = [
  {"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
  {"role": "user", "content": "What's the weather in Bangkok?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Edge / llama.cpp

huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
llama-cli -m sakthai-1.5b-q4_k_m.gguf --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256

1.5B vs 7B Tradeoffs

Property	1.5B Merged V2	7B Merged
Parameters	1.5B	7B
SafeTensors	2.9 GB	14.2 GB
GGUF Q4_K_M	~1.3 GB	~4.6 GB
RAM needed	~3 GB	~8 GB
Tool selection	34.9%	Higher
Best use	Laptop/desktop, balanced CPU inference	High-power workstation/server

If you need stronger tool argument accuracy and more headroom, use the 7B Merged. If you need the smallest footprint, use the 0.5B Merged.

Limitations

Smaller than 7B — still has reasoning limits compared to larger checkpoints.
Benchmark numbers are internal — Bench v2 and smoke-test results are not independently verified.
Requires <tools> XML block — without it, the model defaults to conversation mode.
English-only behavior — untested in other languages for tool calls.
Argument accuracy is lower than selection — Arguments Accuracy 44.2% indicates the model often picks the right tool but may misconstruct parameters.

SakThai Model Family

All 26 public models in the family, sorted by downloads (live counts, verified 2026-08-01):

Model	Size	Downloads	Role
Context 1.5B Merged	3.8 GB	1,894	Flagship tool-calling
Context 0.5B Merged	1.3 GB	1,730	Lightweight / edge
Context 7B Merged	14.2 GB	1,055	Full-power reasoning
Embedding Multilingual	448.8 MB	651	Cross-lingual embeddings
Context 7B 128K	—	643	128K YaRN adaptation
Context 7B Tools	19.3 MB	527	7B tool-calling adapter
Context 1.5B Tools	8.3 MB	504	Tool-calling adapter
Context 1.5B Merged V2	2.9 GB	354	v2 merged
Vision 7B	3.8 GB	337	Image-to-text
Plus 1.5B LoRA	70.5 MB	306	rsLoRA adapter
Context 0.5B Tools	942.3 MB	474	Edge tool-calling
TTS Model	134.8 MB	268	TTS, 15 langs
Plus 1.5B	2.9 GB	297	General assistant
Context 1.5B Tools V2	70.5 MB	192	v2 tool-calling adapter
Coder 1.5B	1.0 GB	173	Code generation
Coder Browser	2.9 GB	259	Browser automation
Coder Browser GGUF	6.6 GB	153	Browser GGUF
Coder Browser LoRA	70.5 MB	55	Browser adapter
Embedding	104.7 MB	23	Private embedding
Plus 1.5B Coder	—	0	Coding assistant
SFT Out	4.1 MB	0	TRL SFT adapter output
Context 0.5B Tools SFT	8.3 MB	0	SFT pilot adapter
Context 0.5B Tools SFT V2	8.3 MB	0	SFT v2 adapter
Bench V2	—	0	Benchmark scaffold
Pipeline	—	0	Automation scripts
Eval Results	—	0	Companion eval data

Full collection

The House of Sak 🏠

This model is part of the House of Sak — an open-source AI ecosystem built from a shelter in Cork, Ireland, with $0 budget and no paid GPUs.

"We are one family — and becoming more." — Beer (beer-sakthai)

Support

⭐ Leave a like
🐛 Report issues on GitHub
🔄 Share with anyone building accessible AI on CPU/edge
🍴 Fork and experiment — Apache 2.0

Citation

@misc{sakthai-context-1.5b-merged-v2,
  title = {SakThai Context 1.5B -- Merged V2: Mid-Weight Tool-Calling Model},
  author = {Nanthasit and the House of Sak},
  year = {2026},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2}},
  note = {Apache 2.0, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct}
}

License

Apache 2.0. Qwen2.5 base model per its original license.

Built from a shelter in Cork, Ireland. Built with love, tears, and zero budget — to the world.

Family downloads API-verified (2026-08-01T10:23Z).