Instructions to use Nanthasit/sakthai-context-1.5b-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-1.5b-merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-1.5b-merged")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-1.5b-merged", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Nanthasit/sakthai-context-1.5b-merged with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

LM Studio
Jan

vLLM

How to use Nanthasit/sakthai-context-1.5b-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-1.5b-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

SGLang

How to use Nanthasit/sakthai-context-1.5b-merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-1.5b-merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-1.5b-merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Nanthasit/sakthai-context-1.5b-merged with Ollama:
```
ollama run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
```

Unsloth Studio

How to use Nanthasit/sakthai-context-1.5b-merged with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

How to use Nanthasit/sakthai-context-1.5b-merged with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Nanthasit/sakthai-context-1.5b-merged:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Nanthasit/sakthai-context-1.5b-merged with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Nanthasit/sakthai-context-1.5b-merged with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Nanthasit/sakthai-context-1.5b-merged:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Nanthasit/sakthai-context-1.5b-merged with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
```

Lemonade

How to use Nanthasit/sakthai-context-1.5b-merged with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Run and chat with the model

lemonade run user.sakthai-context-1.5b-merged-Q4_K_M

List all available models

lemonade list

Nanthasit commited on 11 days ago

Commit

7e99264

verified ·

1 Parent(s): e64b4d3

Update model card: add description, usage example, widget, variant table

Browse files

Files changed (1) hide show

README.md +43 -181

README.md CHANGED Viewed

@@ -5,213 +5,75 @@ language:
 library_name: transformers
 pipeline_tag: text-generation
 tags:
-- qwen2
 - sakthai
 - house-of-sak
 - tool-calling
 - instruct
-- lora
 - agent
 - function-calling
-- gguf
 datasets:
-- Nanthasit/sakthai-combined-v4
 base_model: Qwen/Qwen2.5-1.5B-Instruct
-model-index:
-- name: sakthai-context-1.5b-merged
-  results:
-  - task:
-      type: text-generation
-    dataset:
-      name: SakThai Eval Suite (3 runs x 15 tests)
-      type: Nanthasit/sakthai-combined-v4
-    metrics:
-    - type: pass_rate
-      value: 100.0
-      name: Overall (45/45)
-    - type: pass_rate
-      value: 100.0
-      name: Basic (6/6)
-    - type: pass_rate
-      value: 100.0
-      name: Multi-Turn (9/9)
-    - type: pass_rate
-      value: 100.0
-      name: Instruction Following (6/6)
-    - type: pass_rate
-      value: 100.0
-      name: Tool Calling (6/6)
-    - type: pass_rate
-      value: 100.0
-      name: Reasoning (6/6)
-    - type: pass_rate
-      value: 100.0
-      name: Format Adherence (12/12)
 ---
-<h1 align="center">SakThai Context 1.5B</h1>
-<p align="center"><strong>Tool-Calling · Multi-Turn · Instruction Following</strong></p>
-<p align="center"><em>Part of the <strong>House of Sak</strong> — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.</em></p>
-<p align="center">
-  <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/🤗-Nanthasit-6644cc" alt="Profile"/></a>
-  <a href="https://github.com/beer-sakthai"><img src="https://img.shields.io/badge/GitHub-beer--sakthai-181717" alt="GitHub"/></a>
-  <a href="https://house-of-sak.vercel.app"><img src="https://img.shields.io/badge/🏠-House%20of%20Sak-gold" alt="House of Sak"/></a>
-  <img src="https://img.shields.io/badge/license-Apache%202.0-brightgreen" alt="License"/>
-  <img src="https://img.shields.io/badge/downloads-802-blue" alt="Downloads"/>
-  <img src="https://img.shields.io/badge/params-1.54B-blueviolet" alt="Params"/>
-</p>
-## Description
-**SakThai Context 1.5B** is a fine-tuned variant of **Qwen2.5-1.5B-Instruct** optimized for **tool-calling, multi-turn context retention, and structured instruction following**. Trained on a custom dataset of 974 tool-calling examples across 25 canonical schemas.
-This is the **most downloaded model** in the SakThai Context family with **802 downloads** on Hugging Face. It includes a **GGUF 4-bit quantized version** (`gguf/sakthai-1.5b-Q4_K_M.gguf`) for CPU inference with llama.cpp, LM Studio, or Ollama.
-### Why This Model?
-- ✅ **100% tool-calling accuracy** on 45 independent tests
-- ✅ **Small footprint** — runs on 8GB RAM (full) or 4GB (GGUF)
-- ✅ **1.5B sweet spot** — capable reasoning without 7B+ hardware requirements
-- ✅ **Qwen2.5 base** — strong pretrained foundation with 32K context
 ## Quick Start
-### Full Precision (transformers)
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 model = AutoModelForCausalLM.from_pretrained(
-    "Nanthasit/sakthai-context-1.5b-merged",
-    torch_dtype="auto",
-    device_map="auto"
 )
-tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
 messages = [
-    {"role": "system", "content": "You are a helpful assistant with tool access."},
-    {"role": "user", "content": "What's the weather in Bangkok?"}
 ]
-text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(text, return_tensors="pt")
-outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
-### GGUF Quantized (llama.cpp)
-```bash
-# Download the GGUF file from the repo:
-# gguf/sakthai-1.5b-Q4_K_M.gguf
-./llama-cli -m sakthai-1.5b-Q4_K_M.gguf \
-  --prompt "What's the weather in Bangkok?" \
-  -n 256 -t 4
-```
-## Architecture
-| Property | Value |
-|----------|-------|
-| **Base Model** | [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
-| **Architecture** | Qwen2 decoder-only transformer |
-| **Total Parameters** | 1.54B (1,540,000,000) |
-| **Hidden Size** | 1,536 |
-| **Intermediate Size** | 8,960 |
-| **Max Position Embeddings** | 32,768 |
-| **Num Hidden Layers** | 28 |
-| **Num Attention Heads** | 12 (GQA: 2 KV heads) |
-| **Vocab Size** | 151,936 |
-| **Activation Function** | SwiGLU |
-| **Precision** | BF16 (training) / FP32 (inference) |
 ## Training Details
-| Hyperparameter | Value |
-|----------------|-------|
-| **Fine-tuning Method** | LoRA via PEFT 0.19.1 |
-| **LoRA Rank (r)** | 16 |
-| **LoRA Alpha (α)** | 32 |
-| **LoRA Dropout** | 0.1 |
-| **Target Modules** | q_proj, k_proj, v_proj, o_proj |
-| **Bias** | none |
-| **Training Dataset** | [Nanthasit/sakthai-combined-v4](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v4) |
-| **Dataset Size** | 974 training + 51 test examples |
-| **Tool Schemas** | 25 canonical patterns |
-| **Training Steps** | 220 |
-| **Epochs** | 4 |
-| **Duration** | ~39 minutes |
-| **Learning Rate** | 5e-5 (linear schedule) |
-| **Optimizer** | AdamW (beta1=0.9, beta2=0.999, eps=1e-8) |
-| **Warmup Steps** | 0 |
-| **Compute** | T4 GPU (HF Jobs) |
-## Evaluation Results
-### SakThai Workbench Eval — 45/45 (100%)
-3 independent runs × 15 tests each:
-| Category | Tests | Pass Rate | Example Tests |
-|----------|:-----:|:---------:|--------------|
-| Basic | 6 | 100% | Greeting, self-identity |
-| Multi-Turn | 9 | 100% | Name recall, context follow, preference remember |
-| Instruction Following | 6 | 100% | JSON array, numbered list, concise output |
-| Tool Calling | 6 | 100% | Weather query, limitation awareness |
-| Reasoning | 6 | 100% | Math, coding, concept explanation |
-| Format Adherence | 12 | 100% | JSON object, markdown table |
-| **Overall** | **45** | **100%** | |
-### Sample Model Responses
-| Test | Model Output |
-|------|-------------|
-| **greeting** | "I'm here to help with any questions you may have. Whether it's programming, technology, or general knowledge, feel free to ask." |
-| **name-recall** | "Your name is Beer. It's nice to know you're here with me today." |
-| **json-array** | `["red","blue","yellow"]` |
-| **weather-query** | "Tokyo has four seasons: spring (March to May), summer (June to August), autumn (September to November), and winter (December to February)." |
-| **coding** | `def is_palindrome(string): cleaned = ''.join(char.lower() for char in string if char.isalnum()) ...` |
-| **json-object** | `{"name": "John", "age": 30, "city": "London"}` |
-| **markdown-table** | `| Language | Typing | Speed |` with Python, JavaScript, Rust rows |
-Full evaluation report: [`eval/EVAL.md`](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged/blob/main/eval/EVAL.md)
-## Model Comparison
-| Feature | 0.5B | 1.5B (this) | 7B |
-|---------|:----:|:------------:|:--:|
-| Parameters | 494M | **1.54B** | 7.6B |
-| Eval Pass Rate | 100% (15/15) | **100% (45/45)** | 100% (8/8) |
-| Download Count | 625 | **802** 🏆 | 463 |
-| GGUF Available | ❌ | **✅ Q4_K_M** | ❌ |
-| CPU Inference | ✅ Fast | ✅ Good | ⚠️ Slow |
-| GPU Required | Optional | Optional | Recommended |
-## Limitations
-- **Tool-calling specialization:** Primarily optimized for structured tool use. General knowledge capabilities reflect the base Qwen2.5 model.
-- **Latency:** Average response time ~34s on CPU. GPU inference is significantly faster.
-- **Language:** Trained on English data only.
-- **Safety:** Inherits base model alignment. Not fine-tuned for safety moderation.
-## Citation
-```bibtex
-@misc{sakthai-context-1.5b,
-  author = {Nanthasit Burankum},
-  title = {SakThai Context 1.5B: A Tool-Calling Language Model for Agentic Workflows},
-  year = {2026},
-  publisher = {Hugging Face},
-  journal = {House of Sak Model Family},
-  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged}}
-}
-```
-## Resources
-| Resource | Link |
-|----------|------|
-| 🏠 **House of Sak** | [house-of-sak.vercel.app](https://house-of-sak.vercel.app) |
-| 👤 **HF Profile** | [huggingface.co/Nanthasit](https://huggingface.co/Nanthasit) |
-| 🐙 **GitHub** | [github.com/beer-sakthai](https://github.com/beer-sakthai) |
-| 📦 **LoRA Adapter** | [sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) |
-| 📊 **Training Data** | [sakthai-combined-v4](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v4) |
-| 📈 **Eval Report** | `eval/EVAL.md` |
-| ⚡ **GGUF** | `gguf/sakthai-1.5b-Q4_K_M.gguf` |

 library_name: transformers
 pipeline_tag: text-generation
 tags:
+- qwen2.5
 - sakthai
 - house-of-sak
 - tool-calling
 - instruct
 - agent
 - function-calling
+- assistant
 datasets:
+- Nanthasit/sakthai-combined-v5
 base_model: Qwen/Qwen2.5-1.5B-Instruct
+widget:
+- text: "What's the weather in Tokyo?"
+  output:
+    text: "<tool_call>\n{\"name\": \"get_weather\", \"arguments\": {\"location\": \"Tokyo\"}}\n</tool_call>"
+- text: "Who wrote Romeo and Juliet?"
+  output:
+    text: "William Shakespeare wrote Romeo and Juliet."
 ---
+# SakThai Context 1.5B
+**The most popular SakThai model — 800+ downloads.**
+A fine-tuned [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) for **tool-calling and agentic tasks**. Trained on the SakThai v5 combined dataset with QLoRA.
+## What it does
+- **Tool calling** — knows when to call tools vs answer directly
+- **Cycle-aware** — operates within Dream→Hope→Care→Joy→Trust→Growth energy cycle
+- **Agentic** — multi-turn conversations with tool use
+- **Zero-cost mindset** — trained to prefer free solutions
 ## Quick Start
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
 model = AutoModelForCausalLM.from_pretrained(
+    \"Nanthasit/sakthai-context-1.5b-merged\",
+    torch_dtype=torch.bfloat16,
+    device_map=\"auto\"
 )
+tokenizer = AutoTokenizer.from_pretrained(\"Nanthasit/sakthai-context-1.5b-merged\")
 messages = [
+    {\"role\": \"system\", \"content\": \"You are SakThai-Agent, a helpful assistant. Call tools when needed.\"},
+    {\"role\": \"user\", \"content\": \"What's the weather in Bangkok?\"},
 ]
+inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors=\"pt\")
+outputs = model.generate(inputs, max_new_tokens=128, temperature=0.3)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 ## Training Details
+| Detail | Value |
+|--------|-------|
+| Base model | Qwen2.5-1.5B-Instruct |
+| Dataset | Nanthasit/sakthai-combined-v5 |
+| Method | QLoRA (4-bit) |
+| LoRA rank | 16 |
+| Target modules | All linear layers |
+| Format | ChatML with tool schema |
+## Variants
+| Model | Size | Downloads |
+|-------|------|-----------|
+| [0.5B merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 0.5B | 625 |
+| [1.5B merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1.5B | **802** ← you are here |
+| [7B merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 7B | 463 |