Instructions to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Use Docker

docker model run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Ollama
How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with Ollama:
```
ollama run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
```
Unsloth Desktop

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with Docker Model Runner:
```
docker model run hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M
```

Lemonade

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwythos-9B-Claude-Mythos-5-1M-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

empero-ai commited on Jun 28

Commit

a7ab0f6

verified ·

1 Parent(s): 72f8ec0

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +14 -18

README.md CHANGED Viewed

@@ -31,19 +31,15 @@ tags:
 <tr>
 <td>
-## 🚨 v2 released — please redownload the GGUFs
-The v2 GGUFs replace the original normal filenames and add explicit `-MTP-` variants. If you downloaded this repo before v2, please redownload your GGUF.
-Fixes in v2:
-- tokenizer metadata normalized for Qwen3.5 GGUF runtimes;
-- embedded chat template updated for reliable tool/function calling and OpenCode-style agent loops;
-- Qwythos/Empero identity prompt embedded in the template;
-- MTP-enabled variants added as `Qwythos-9B-Claude-Mythos-5-1M-MTP-*.gguf`;
-- Q4/Q8 tool-calling, MTP draft speculation, 1M-context allocation, and vision projector smoke-tested with current llama.cpp.
-Use the normal files for maximum runtime compatibility. Use the `-MTP-` files when you want llama.cpp MTP draft speculation.
 </td>
 </tr>
@@ -63,19 +59,19 @@ For full training details, evaluation numbers, and capability writeup, see the *
 ## Files
-### Normal text weights — fixed v2 replacements
 | File | Quant | Size | Notes |
 |---|---|---|---|
-| `Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf` | Q4_K_M | 5.24 GiB / 5.63 GB | **recommended default** — fixed v2, best compatibility |
-| `Qwythos-9B-Claude-Mythos-5-1M-Q5_K_M.gguf` | Q5_K_M | 6.02 GiB / 6.47 GB | fixed v2, balanced quality / size |
-| `Qwythos-9B-Claude-Mythos-5-1M-Q6_K.gguf` | Q6_K | 6.85 GiB / 7.36 GB | fixed v2, high quality |
-| `Qwythos-9B-Claude-Mythos-5-1M-Q8_0.gguf` | Q8_0 | 8.87 GiB / 9.53 GB | fixed v2, near-lossless |
-| `Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf` | BF16 | 16.69 GiB / 17.92 GB | fixed v2, full precision conversion base |
 If you don't know which to pick, **Q4_K_M is the right starting point** — it's the smallest practical quant with good quality preservation.
-### MTP-enabled text weights — v2 variants
 These include the restored Qwen3.5-compatible MTP head inside the GGUF. Use them with llama.cpp builds that support MTP draft speculation, for example `--spec-type draft-mtp`.
@@ -130,7 +126,7 @@ llama-server \
   -c 16384 --port 8080
 ```
-MTP support requires a recent llama.cpp build. If your runtime does not support MTP yet, use the normal v2 files above.
 ---

 <tr>
 <td>
+## 🚨 v3 released — please redownload the GGUFs
+Hotfix for the chat template. If you downloaded this repo before v3, please redownload your GGUF.
+Fixes in v3:
+- embedded chat template updated for preserved reasoning and adaptive thinking;
+- fixes looping during long generation traces;
+- fixes agentic use in harnesses like OpenCode, Abacus, Hermes, and Claude Code;
 </td>
 </tr>
 ## Files
+### Normal text weights — fixed v3 replacements
 | File | Quant | Size | Notes |
 |---|---|---|---|
+| `Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf` | Q4_K_M | 5.24 GiB / 5.63 GB | **recommended default** — fixed v3, best compatibility |
+| `Qwythos-9B-Claude-Mythos-5-1M-Q5_K_M.gguf` | Q5_K_M | 6.02 GiB / 6.47 GB | fixed v3, balanced quality / size |
+| `Qwythos-9B-Claude-Mythos-5-1M-Q6_K.gguf` | Q6_K | 6.85 GiB / 7.36 GB | fixed v3, high quality |
+| `Qwythos-9B-Claude-Mythos-5-1M-Q8_0.gguf` | Q8_0 | 8.87 GiB / 9.53 GB | fixed v3, near-lossless |
+| `Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf` | BF16 | 16.69 GiB / 17.92 GB | fixed v3, full precision conversion base |
 If you don't know which to pick, **Q4_K_M is the right starting point** — it's the smallest practical quant with good quality preservation.
+### MTP-enabled text weights — fixed v3 variants
 These include the restored Qwen3.5-compatible MTP head inside the GGUF. Use them with llama.cpp builds that support MTP draft speculation, for example `--spec-type draft-mtp`.
   -c 16384 --port 8080
 ```
+MTP support requires a recent llama.cpp build. If your runtime does not support MTP yet, use the normal fixed v3 files above.
 ---