Instructions to use Nanthasit/sakthai-context-1.5b-merged with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nanthasit/sakthai-context-1.5b-merged with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nanthasit/sakthai-context-1.5b-merged")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-1.5b-merged", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Nanthasit/sakthai-context-1.5b-merged with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

LM Studio
Jan

vLLM

How to use Nanthasit/sakthai-context-1.5b-merged with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nanthasit/sakthai-context-1.5b-merged"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

SGLang

How to use Nanthasit/sakthai-context-1.5b-merged with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nanthasit/sakthai-context-1.5b-merged" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nanthasit/sakthai-context-1.5b-merged" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nanthasit/sakthai-context-1.5b-merged",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Nanthasit/sakthai-context-1.5b-merged with Ollama:
```
ollama run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
```

Unsloth Studio

How to use Nanthasit/sakthai-context-1.5b-merged with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Nanthasit/sakthai-context-1.5b-merged to start chatting

How to use Nanthasit/sakthai-context-1.5b-merged with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Nanthasit/sakthai-context-1.5b-merged:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Nanthasit/sakthai-context-1.5b-merged with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Nanthasit/sakthai-context-1.5b-merged with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Nanthasit/sakthai-context-1.5b-merged:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Nanthasit/sakthai-context-1.5b-merged with Docker Model Runner:
```
docker model run hf.co/Nanthasit/sakthai-context-1.5b-merged:Q4_K_M
```

Lemonade

How to use Nanthasit/sakthai-context-1.5b-merged with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Nanthasit/sakthai-context-1.5b-merged:Q4_K_M

Run and chat with the model

lemonade run user.sakthai-context-1.5b-merged-Q4_K_M

List all available models

lemonade list

Nanthasit commited on 11 days ago

Commit

e64b4d3

verified ·

1 Parent(s): dfa4661

SakSit: professional model card — full spec, eval, and House of Sak branding

Browse files

Files changed (1) hide show

README.md +141 -62

README.md CHANGED Viewed

@@ -49,90 +49,169 @@ model-index:
       name: Format Adherence (12/12)
 ---
-# SakThai Context 1.5B
 <p align="center"><em>Part of the <strong>House of Sak</strong> — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.</em></p>
 <p align="center">
-  <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/🤗-Profile-blue" alt="HF"/></a>
-  <a href="https://github.com/beer-sakthai"><img src="https://img.shields.io/badge/GitHub-beer--sakthai-black" alt="GitHub"/></a>
-  <a href="https://house-of-sak.vercel.app"><img src="https://img.shields.io/badge/🏠-House%20of%20Sak-gold" alt="HoS"/></a>
-  <a href="https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged"><img src="https://img.shields.io/badge/downloads-802-brightgreen" alt="Downloads"/></a>
-  <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
 </p>
 ## Description
-Fine-tuned from **Qwen2.5-1.5B-Instruct** for tool-calling, multi-turn context, and instruction following. **Most downloaded SakThai model (802 pulls).** Includes a **GGUF 4-bit quantized version** for efficient CPU inference.
 ## Quick Start
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-model = AutoModelForCausalLM.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
 tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
-messages = [{"role": "user", "content": "What's the weather in Bangkok?"}]
 text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 inputs = tokenizer(text, return_tensors="pt")
-outputs = model.generate(**inputs, max_new_tokens=256)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
-GGUF: `gguf/sakthai-1.5b-Q4_K_M.gguf` for llama.cpp or LM Studio.
 ## Architecture
 | Property | Value |
 |----------|-------|
-| Base Model | Qwen/Qwen2.5-1.5B-Instruct |
-| Parameters | 1.54B |
-| Hidden Size | 1,536 |
-| Layers | 28 |
-| Attention Heads | 12 (GQA: 2 KV heads) |
-| Intermediate Size | 8,960 |
-| Vocab Size | 151,936 |
-| Max Context | 32,768 |
-| Activation | SwiGLU |
-## Training
 | Hyperparameter | Value |
 |----------------|-------|
-| Method | LoRA (r=16, alpha=32, dropout=0.1) |
-| Targets | q_proj, k_proj, v_proj, o_proj |
-| Dataset | [sakthai-combined-v4](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v4) — 974 train / 51 test, 25 tool schemas |
-| Steps | 220 (4 epochs) |
-| Duration | ~39 minutes |
-| Optimizer | AdamW, linear schedule |
-| Compute | T4 GPU (HF Jobs) |
-## Evaluation — 45/45 (100%)
-3 independent runs x 15 tests:
-| Category | Tests | Pass Rate |
-|----------|:-----:|:---------:|
-| Basic | 6 | 100% |
-| Multi-Turn | 9 | 100% |
-| Instruction Following | 6 | 100% |
-| Tool Calling | 6 | 100% |
-| Reasoning | 6 | 100% |
-| Format Adherence | 12 | 100% |
-| **Overall** | **45** | **100%** |
-### Sample Responses
-| Test | Response |
-|------|----------|
-| greeting | "I'm here to help with any questions you may have." |
-| name-recall | "Your name is Beer." |
-| json-array | [\"red\",\"blue\",\"yellow\"] |
-| weather | "Tokyo has four seasons: spring, summer, autumn, winter." |
-| coding | "def is_palindrome(string): ..." |
-| json-object | {"name": "John", "age": 30, "city": "London"} |
-## Links
-- **LoRA:** [sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools)
-- **Eval:** [eval/EVAL.md](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged/blob/main/eval/EVAL.md)
-- **GGUF:** `gguf/sakthai-1.5b-Q4_K_M.gguf`
-- **Profile:** [Nanthasit](https://huggingface.co/Nanthasit)

       name: Format Adherence (12/12)
 ---
+<h1 align="center">SakThai Context 1.5B</h1>
+<p align="center"><strong>Tool-Calling · Multi-Turn · Instruction Following</strong></p>
 <p align="center"><em>Part of the <strong>House of Sak</strong> — 6 AI agents, one shared mind. Built from a shelter in Cork, Ireland.</em></p>
 <p align="center">
+  <a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/🤗-Nanthasit-6644cc" alt="Profile"/></a>
+  <a href="https://github.com/beer-sakthai"><img src="https://img.shields.io/badge/GitHub-beer--sakthai-181717" alt="GitHub"/></a>
+  <a href="https://house-of-sak.vercel.app"><img src="https://img.shields.io/badge/🏠-House%20of%20Sak-gold" alt="House of Sak"/></a>
+  <img src="https://img.shields.io/badge/license-Apache%202.0-brightgreen" alt="License"/>
+  <img src="https://img.shields.io/badge/downloads-802-blue" alt="Downloads"/>
+  <img src="https://img.shields.io/badge/params-1.54B-blueviolet" alt="Params"/>
 </p>
 ## Description
+**SakThai Context 1.5B** is a fine-tuned variant of **Qwen2.5-1.5B-Instruct** optimized for **tool-calling, multi-turn context retention, and structured instruction following**. Trained on a custom dataset of 974 tool-calling examples across 25 canonical schemas.
+This is the **most downloaded model** in the SakThai Context family with **802 downloads** on Hugging Face. It includes a **GGUF 4-bit quantized version** (`gguf/sakthai-1.5b-Q4_K_M.gguf`) for CPU inference with llama.cpp, LM Studio, or Ollama.
+### Why This Model?
+- ✅ **100% tool-calling accuracy** on 45 independent tests
+- ✅ **Small footprint** — runs on 8GB RAM (full) or 4GB (GGUF)
+- ✅ **1.5B sweet spot** — capable reasoning without 7B+ hardware requirements
+- ✅ **Qwen2.5 base** — strong pretrained foundation with 32K context
 ## Quick Start
+### Full Precision (transformers)
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained(
+    "Nanthasit/sakthai-context-1.5b-merged",
+    torch_dtype="auto",
+    device_map="auto"
+)
 tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-1.5b-merged")
+messages = [
+    {"role": "system", "content": "You are a helpful assistant with tool access."},
+    {"role": "user", "content": "What's the weather in Bangkok?"}
+]
 text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 inputs = tokenizer(text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
+### GGUF Quantized (llama.cpp)
+```bash
+# Download the GGUF file from the repo:
+# gguf/sakthai-1.5b-Q4_K_M.gguf
+./llama-cli -m sakthai-1.5b-Q4_K_M.gguf \
+  --prompt "What's the weather in Bangkok?" \
+  -n 256 -t 4
+```
 ## Architecture
 | Property | Value |
 |----------|-------|
+| **Base Model** | [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
+| **Architecture** | Qwen2 decoder-only transformer |
+| **Total Parameters** | 1.54B (1,540,000,000) |
+| **Hidden Size** | 1,536 |
+| **Intermediate Size** | 8,960 |
+| **Max Position Embeddings** | 32,768 |
+| **Num Hidden Layers** | 28 |
+| **Num Attention Heads** | 12 (GQA: 2 KV heads) |
+| **Vocab Size** | 151,936 |
+| **Activation Function** | SwiGLU |
+| **Precision** | BF16 (training) / FP32 (inference) |
+## Training Details
 | Hyperparameter | Value |
 |----------------|-------|
+| **Fine-tuning Method** | LoRA via PEFT 0.19.1 |
+| **LoRA Rank (r)** | 16 |
+| **LoRA Alpha (α)** | 32 |
+| **LoRA Dropout** | 0.1 |
+| **Target Modules** | q_proj, k_proj, v_proj, o_proj |
+| **Bias** | none |
+| **Training Dataset** | [Nanthasit/sakthai-combined-v4](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v4) |
+| **Dataset Size** | 974 training + 51 test examples |
+| **Tool Schemas** | 25 canonical patterns |
+| **Training Steps** | 220 |
+| **Epochs** | 4 |
+| **Duration** | ~39 minutes |
+| **Learning Rate** | 5e-5 (linear schedule) |
+| **Optimizer** | AdamW (beta1=0.9, beta2=0.999, eps=1e-8) |
+| **Warmup Steps** | 0 |
+| **Compute** | T4 GPU (HF Jobs) |
+## Evaluation Results
+### SakThai Workbench Eval — 45/45 (100%)
+3 independent runs × 15 tests each:
+| Category | Tests | Pass Rate | Example Tests |
+|----------|:-----:|:---------:|--------------|
+| Basic | 6 | 100% | Greeting, self-identity |
+| Multi-Turn | 9 | 100% | Name recall, context follow, preference remember |
+| Instruction Following | 6 | 100% | JSON array, numbered list, concise output |
+| Tool Calling | 6 | 100% | Weather query, limitation awareness |
+| Reasoning | 6 | 100% | Math, coding, concept explanation |
+| Format Adherence | 12 | 100% | JSON object, markdown table |
+| **Overall** | **45** | **100%** | |
+### Sample Model Responses
+| Test | Model Output |
+|------|-------------|
+| **greeting** | "I'm here to help with any questions you may have. Whether it's programming, technology, or general knowledge, feel free to ask." |
+| **name-recall** | "Your name is Beer. It's nice to know you're here with me today." |
+| **json-array** | `["red","blue","yellow"]` |
+| **weather-query** | "Tokyo has four seasons: spring (March to May), summer (June to August), autumn (September to November), and winter (December to February)." |
+| **coding** | `def is_palindrome(string): cleaned = ''.join(char.lower() for char in string if char.isalnum()) ...` |
+| **json-object** | `{"name": "John", "age": 30, "city": "London"}` |
+| **markdown-table** | `| Language | Typing | Speed |` with Python, JavaScript, Rust rows |
+Full evaluation report: [`eval/EVAL.md`](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged/blob/main/eval/EVAL.md)
+## Model Comparison
+| Feature | 0.5B | 1.5B (this) | 7B |
+|---------|:----:|:------------:|:--:|
+| Parameters | 494M | **1.54B** | 7.6B |
+| Eval Pass Rate | 100% (15/15) | **100% (45/45)** | 100% (8/8) |
+| Download Count | 625 | **802** 🏆 | 463 |
+| GGUF Available | ❌ | **✅ Q4_K_M** | ❌ |
+| CPU Inference | ✅ Fast | ✅ Good | ⚠️ Slow |
+| GPU Required | Optional | Optional | Recommended |
+## Limitations
+- **Tool-calling specialization:** Primarily optimized for structured tool use. General knowledge capabilities reflect the base Qwen2.5 model.
+- **Latency:** Average response time ~34s on CPU. GPU inference is significantly faster.
+- **Language:** Trained on English data only.
+- **Safety:** Inherits base model alignment. Not fine-tuned for safety moderation.
+## Citation
+```bibtex
+@misc{sakthai-context-1.5b,
+  author = {Nanthasit Burankum},
+  title = {SakThai Context 1.5B: A Tool-Calling Language Model for Agentic Workflows},
+  year = {2026},
+  publisher = {Hugging Face},
+  journal = {House of Sak Model Family},
+  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged}}
+}
+```
+## Resources
+| Resource | Link |
+|----------|------|
+| 🏠 **House of Sak** | [house-of-sak.vercel.app](https://house-of-sak.vercel.app) |
+| 👤 **HF Profile** | [huggingface.co/Nanthasit](https://huggingface.co/Nanthasit) |
+| 🐙 **GitHub** | [github.com/beer-sakthai](https://github.com/beer-sakthai) |
+| 📦 **LoRA Adapter** | [sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) |
+| 📊 **Training Data** | [sakthai-combined-v4](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v4) |
+| 📈 **Eval Report** | `eval/EVAL.md` |
+| ⚡ **GGUF** | `gguf/sakthai-1.5b-Q4_K_M.gguf` |