Instructions to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF", dtype="auto")

llama-cpp-python

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF",
	filename="Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-BF16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Use Docker

docker model run hf.co/llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

SGLang

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Ollama:
```
ollama run hf.co/llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
```

Unsloth Studio

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF to start chatting

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Docker Model Runner:
```
docker model run hf.co/llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M
```

Lemonade

How to use llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull llmfan46/Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF-Q4_K_M

List all available models

lemonade list

Qwythos-9B-Claude-Mythos-5-1M-uncensored-heretic-GGUF / assets /qwythos_eval_chart.svg

llmfan46

Upload 2 files

f1750b6 verified 3 days ago

preview code

Raw

History Blame Contribute Delete

6.78 kB

	<svg viewBox="0 0 1000 680" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Qwythos evaluation results versus base Qwen3.5-9B across seven benchmarks">
	<title>Qwythos evaluation results vs. base Qwen3.5-9B</title>
	<desc>Grouped horizontal bar chart. gsm8k flex 0.670 to 0.860 (+0.19); gsm8k strict 0.510 to 0.810 (+0.30); gpqa_diamond flex 0.630 to 0.580 (-0.05); gpqa_diamond strict 0.050 to 0.010 (-0.04); mmlu 0.232 to 0.575 (+0.34); arc_challenge acc 0.470 to 0.490 (+0.02); arc_challenge acc_norm 0.400 to 0.410 (+0.01).</desc>
	<defs>
	<linearGradient id="qwgrad" gradientUnits="userSpaceOnUse" x1="280" y1="0" x2="800" y2="0">
	<stop offset="0" stop-color="#2dd4bf"/>
	<stop offset="0.55" stop-color="#56b6e6"/>
	<stop offset="1" stop-color="#8b7ff0"/>
	</linearGradient>
	<filter id="glow" x="-20%" y="-80%" width="140%" height="260%">
	<feGaussianBlur stdDeviation="4.5"/>
	</filter>
	</defs>

	<rect x="0" y="0" width="1000" height="680" fill="#0a0c11"/>

	<text x="40" y="58" font-family="Georgia, 'Times New Roman', serif" font-size="34" fill="#f5f3ee">Qwythos</text>
	<text x="42" y="86" font-family="Georgia, 'Times New Roman', serif" font-size="15" fill="#9b9892">Evaluation results — vs. base Qwen3.5-9B</text>

	<g font-family="system-ui, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="12.5" fill="#9b9892">
	<rect x="636" y="46" width="13" height="13" rx="2" fill="#41454e"/>
	<text x="656" y="57">Base Qwen3.5-9B</text>
	<rect x="772" y="46" width="13" height="13" rx="2" fill="url(#qwgrad)"/>
	<text x="792" y="57">Qwythos</text>
	</g>

	<g stroke="#ffffff" stroke-opacity="0.06" stroke-width="1">
	<line x1="280" y1="172" x2="280" y2="612"/>
	<line x1="384" y1="172" x2="384" y2="612"/>
	<line x1="488" y1="172" x2="488" y2="612"/>
	<line x1="592" y1="172" x2="592" y2="612"/>
	<line x1="696" y1="172" x2="696" y2="612"/>
	<line x1="800" y1="172" x2="800" y2="612"/>
	</g>
	<g font-family="ui-monospace, 'SF Mono', Menlo, monospace" font-size="11" fill="#6f6d68" text-anchor="middle">
	<text x="280" y="630">0.0</text>
	<text x="384" y="630">0.2</text>
	<text x="488" y="630">0.4</text>
	<text x="592" y="630">0.6</text>
	<text x="696" y="630">0.8</text>
	<text x="800" y="630">1.0</text>
	</g>

	<text x="835" y="165" font-family="system-ui, sans-serif" font-size="12.5" fill="#9b9892">Δ vs. base</text>

	<g filter="url(#glow)" opacity="0.45">
	<rect x="280" y="223" width="447.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="285" width="421.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="347" width="301.6" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="409" width="5.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="471" width="299" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="533" width="254.8" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="595" width="213.2" height="14" rx="2" fill="url(#qwgrad)"/>
	</g>

	<g font-family="system-ui, -apple-system, 'Segoe UI', Helvetica, Arial, sans-serif" font-size="14" fill="#c9c6be" text-anchor="end">
	<text x="262" y="219">gsm8k (flex)</text>
	<text x="262" y="281">gsm8k (strict)</text>
	<text x="262" y="343">gpqa_diamond (flex)</text>
	<text x="262" y="405">gpqa_diamond (strict)</text>
	<text x="262" y="467">mmlu (acc)</text>
	<text x="262" y="529">arc_challenge (acc)</text>
	<text x="262" y="591">arc_challenge (acc_norm)</text>
	</g>

	<g>
	<rect x="280" y="201" width="348.4" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="223" width="447.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="263" width="265.2" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="285" width="421.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="325" width="327.6" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="347" width="301.6" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="387" width="26" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="409" width="5.2" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="449" width="120.64" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="471" width="299" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="511" width="244.4" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="533" width="254.8" height="14" rx="2" fill="url(#qwgrad)"/>
	<rect x="280" y="573" width="208" height="14" rx="2" fill="#41454e"/>
	<rect x="280" y="595" width="213.2" height="14" rx="2" fill="url(#qwgrad)"/>
	</g>

	<g font-family="ui-monospace, 'SF Mono', Menlo, monospace" font-size="11">
	<text x="636.4" y="212" fill="#8a8d95">0.670</text>
	<text x="735.2" y="234" fill="#e9e7e1">0.860</text>
	<text x="553.2" y="274" fill="#8a8d95">0.510</text>
	<text x="709.2" y="296" fill="#e9e7e1">0.810</text>
	<text x="615.6" y="336" fill="#8a8d95">0.630</text>
	<text x="589.6" y="358" fill="#e9e7e1">0.580</text>
	<text x="314" y="398" fill="#8a8d95">0.050</text>
	<text x="293.2" y="420" fill="#e9e7e1">0.010</text>
	<text x="408.6" y="460" fill="#8a8d95">0.232</text>
	<text x="587" y="482" fill="#e9e7e1">0.575</text>
	<text x="532.4" y="522" fill="#8a8d95">0.470</text>
	<text x="542.8" y="544" fill="#e9e7e1">0.490</text>
	<text x="496" y="584" fill="#8a8d95">0.400</text>
	<text x="501.2" y="606" fill="#e9e7e1">0.410</text>
	</g>

	<g font-family="ui-monospace, 'SF Mono', Menlo, monospace" font-weight="bold">
	<text x="835" y="215" font-size="17" fill="#4ade80">+0.19</text>
	<text x="835" y="277" font-size="18" fill="#4ade80">+0.30</text>
	<text x="835" y="339" font-size="17" fill="#f08a90">−0.05</text>
	<text x="835" y="401" font-size="17" fill="#f08a90">−0.04</text>
	<text x="835" y="463" font-size="19" fill="#4ade80">+0.34</text>
	<text x="835" y="525" font-size="16" fill="#7faf93">+0.02</text>
	<text x="835" y="587" font-size="16" fill="#7faf93">+0.01</text>
	</g>
	<g font-family="system-ui, sans-serif" font-size="9.5" letter-spacing="1.2">
	<text x="835" y="229" fill="#6f9a7e">BIG WIN</text>
	<text x="835" y="291" fill="#6f9a7e">HUGE</text>
	<text x="835" y="353" fill="#b08084">SMALL LOSS</text>
	<text x="835" y="415" fill="#8c8a85">FORMAT NOISE</text>
	<text x="835" y="477" fill="#6f9a7e">ENORMOUS</text>
	<text x="835" y="539" fill="#6f6d68">WITHIN NOISE</text>
	<text x="835" y="601" fill="#6f6d68">WITHIN NOISE</text>
	</g>

	<text x="40" y="656" font-family="ui-monospace, 'SF Mono', Menlo, monospace" font-size="11" fill="#55534e">plate · qwythos eval-v1 · 7 tasks · accuracy 0–1</text>
	</svg>