Instructions to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT",
	filename="mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00001-of-00024.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
# Run inference directly in the terminal:
llama-cli -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
# Run inference directly in the terminal:
llama-cli -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
# Run inference directly in the terminal:
./llama-cli -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Use Docker

docker model run hf.co/Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

LM Studio
Jan
Ollama
How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Ollama:
```
ollama run hf.co/Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
```

Unsloth Studio

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT to start chatting

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Docker Model Runner:
```
docker model run hf.co/Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT
```

Lemonade

How to use Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Thireus/mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

Run and chat with the model

lemonade run user.mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT-{{QUANT_TAG}}

List all available models

lemonade list

mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT / tensors.map

Thireus

mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_SPLIT

9a3044d 29 days ago

Raw

History Blame Contribute Delete

5.11 kB

	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00002-of-00024.gguf:d76dee061910b49588b5d336ace7df3b631a10f8f782c0ec6eae963b915f1dae:output.weight:shape=(2048, 248320):dtype=iq5_k_r4:elements=508559360:bytes=349634560
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00003-of-00024.gguf:c690e499053346d43a736745ca9e8e97570989d61422b6e94c2e1da71c4e21c5:token_embd.weight:shape=(2048, 248320):dtype=iq5_k:elements=508559360:bytes=349634560
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00004-of-00024.gguf:fdfcc9b9b4fa1c309cd1878499e466c039ffa69cc190f4220ddb71eb39675923:blk.40.nextn.eh_proj.weight:shape=(4096, 2048):dtype=iq5_k_r4:elements=8388608:bytes=5767168
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00005-of-00024.gguf:badbd67d959d5168b2e4320c2fb5fee2c21a0a15355a4bbaba06d9085a69ff9b:blk.40.attn_output.weight:shape=(4096, 2048):dtype=iq5_k_r4:elements=8388608:bytes=5767168
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00006-of-00024.gguf:8b4f71a6434768d27bca8213edec6004ce122cfd8bcfedfb75b668102945ea52:blk.40.attn_q.weight:shape=(2048, 8192):dtype=iq5_k_r4:elements=16777216:bytes=11534336
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00007-of-00024.gguf:c5be725838e806331bae127a6bcc3ac3f2f680b3df82e515048fdd0a419d59ea:output_norm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00008-of-00024.gguf:08beb9c6e7cd3a1acff6b4090ea320030bcdf4c0dec9be598fca725fe20fe407:blk.40.attn_norm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00009-of-00024.gguf:4ad1e82966a9a49243447603624ef4fee13def5389afcafea4dd314f7421abbb:blk.40.ffn_down_exps.weight:shape=(512, 2048, 256):dtype=iq5_k_r4:elements=268435456:bytes=184549376
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00010-of-00024.gguf:1da34bd2b0cd681760e343b6ba6a1cd8a3f539bdbaef929888dcbc46fcb626e8:blk.40.ffn_gate_exps.weight:shape=(2048, 512, 256):dtype=iq5_k_r4:elements=268435456:bytes=184549376
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00011-of-00024.gguf:3434f4c4efdd33d8da58e1a0d3ec4e0da7546471e627248a7759395885c06d00:blk.40.ffn_up_exps.weight:shape=(2048, 512, 256):dtype=iq5_k_r4:elements=268435456:bytes=184549376
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00012-of-00024.gguf:b0df2b2fac171285aa30dfab04178fa7364559ea78e460291e7c3cb1236ebc58:blk.40.ffn_gate_inp.weight:shape=(2048, 256):dtype=bf16:elements=524288:bytes=1048576
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00013-of-00024.gguf:199e7cd3464994379d427cd1ac7d95d7779105759a03660251eba952a3ff89fe:blk.40.ffn_down_shexp.weight:shape=(512, 2048):dtype=iq5_k_r4:elements=1048576:bytes=720896
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00014-of-00024.gguf:7ee114eb8caa46f0fefa183c526376e627ad70bb768f8e552a2500ce8991c95b:blk.40.ffn_gate_shexp.weight:shape=(2048, 512):dtype=iq5_k_r4:elements=1048576:bytes=720896
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00015-of-00024.gguf:6b7dc6a8507e0447aac97f5ded7aa85ea32d3f190bf7434a0207abef572d7f6e:blk.40.ffn_up_shexp.weight:shape=(2048, 512):dtype=iq5_k_r4:elements=1048576:bytes=720896
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00016-of-00024.gguf:3fc254211402f7e451d19990392d7a912ddaaed128aff56a44fd47094d73e450:blk.40.ffn_gate_inp_shexp.weight:shape=(2048,):dtype=bf16:elements=2048:bytes=4096
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00017-of-00024.gguf:05f1430d54301a4f21f6fb4221e7fd7276284136886549c172a66cee3310d770:blk.40.post_attention_norm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00018-of-00024.gguf:ab43e7c05f7055fcf1afbaf9bb7ccdeac21d22cd706460bb7ec157e7a696158b:blk.40.attn_k_norm.weight:shape=(256,):dtype=f32:elements=256:bytes=1024
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00019-of-00024.gguf:a2710424d996d2fb66e8e815f4fd9cdacc4be83b69c71048ec9570594fe26f64:blk.40.attn_k.weight:shape=(2048, 512):dtype=iq5_k_r4:elements=1048576:bytes=720896
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00020-of-00024.gguf:125d7b98d544e743112c447d6c1fbb29192a3fb42dfaff04b6deaff4d2148aca:blk.40.attn_q_norm.weight:shape=(256,):dtype=f32:elements=256:bytes=1024
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00021-of-00024.gguf:2da524497a2052387fa23d9cd07da7625806b41d582f5678dcdb45a1fad7c6a6:blk.40.attn_v.weight:shape=(2048, 512):dtype=iq5_k_r4:elements=1048576:bytes=720896
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00022-of-00024.gguf:1e5489650f37c6b333039b1511170a6b339f52418c2c78b0b93b015e8046d53a:blk.40.nextn.shared_head_norm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00023-of-00024.gguf:c1e6023e08d7b91a3c4daeda24497868e1c98db437b4868b5246bc6e00a6f23b:blk.40.nextn.enorm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192
	mtp-Qwen3.5-35B-A3B-THIREUS-IQ5_K_R4-SPECIAL_TENSOR-00024-of-00024.gguf:017b7d8a8774eeeb7458061453f3a2d39d56c1b30fc9ecba739ed65741be03cc:blk.40.nextn.hnorm.weight:shape=(2048,):dtype=f32:elements=2048:bytes=8192