Instructions to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
# Run inference directly in the terminal:
llama cli -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
# Run inference directly in the terminal:
./llama-cli -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Use Docker

docker model run hf.co/deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

LM Studio
Jan

vLLM

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Ollama
How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with Ollama:
```
ollama run hf.co/deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
```
Unsloth Desktop

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with Docker Model Runner:
```
docker model run hf.co/deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M
```

Lemonade

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF-Q3_K_M

List all available models

lemonade list

Hermes Agent

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "deucebucket/Qwen3.6-35B-A3B-Heretic-Cerebellum-GGUF:Q3_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Good results and possible request

by NeoHuggingF - opened 30 days ago

Discussion

NeoHuggingF

30 days ago

•

edited 30 days ago

Hi, there!

First of all, thank you for your work and the weights you provide on HF. I have tested your Gemma4 26B Uncensored model and it was the best I have found so far for general math, logic, basic coding (mostly front end) while also being uncensored. Unfortunately the Qwen3.6 versions did not gave me good results in my private basic math and logic tests (I testes both uncensored and not, as well as the bigger versions), maybe because:

Bigger reduction in size when compared to the Gemma4 models
I would say that Qwen3.x models are much more sensitive to fine -tuning/obliteration

This leads to my request: try to use Cerebellum with Kat-Coder-V2.5, which is a fine tuning of Qwen3.6 35B-A3B, so perhaps it is simpler to apply. This model seems to perform better in general coding/agentic tasks, with much lower "thinking" tokens output. Just a thought...

Thanks again. Regards

deucebucket

Owner 29 days ago

Hey, thanks! and thanks for reporting back, it helps.

yeah for the qwen i was trying to find the floor, but never really explored back that other way yet. The 27b with thinking off, isnt bad if you havent tried it. Gemma was also trained to be squashed so it takes a couple more dents to the head before thought collapses.

And for the kat-coder-v2.5 is machers the 35b-a3b base so it should just be an apply and bench. I have it on the list. I'll reply back here once I have an updated.

Thanks again for trying it out!

NeoHuggingF

29 days ago

•

edited 29 days ago

Hi, again. Thank you a lot for your quick reply. I forgot to add a few points stemming from my private and nondeterministic tests:

Base Qwen3.6 generally performs much better in tests that probably weren't included in the training data because of the very long "thinking" tokens output. I say this because other models that were based on them that claim to output fewer tokens actually perform worse in those prompts. By the other hand, they perform better in more "deterministic" tasks like coding and agentic tasks (this goes for Nex-N2-mini, Ornith and Kat-Coder-v2.5), exactly the tasks that Qwen3.6 was "benchmaxed", and fewer tokens may actually help in answering all tests in the benchmark. Unfortunately I do not have the hardware required to run dense models like 12B and above in a reasonable speed, but my guess is that this affects dense models less, this is why I think BottleCapAI released a Thinkingcap version of 27B but not for the 35B MOE.
Just like the cap in "thinking" tokens, making a model obliterated/uncensored also hurt their ability to "self-correct" when they are doing the "chain of thought" (CoT). I have seen others say this too, that removing the guardrails that would make the model uncensored also affect their CoT, specially for logic/math prompts. This can also be seen when using the same math/logic prompts with those uncensored models with "thinking" disabled, the number of correct answers drops a lot. For this reason, for coding and agentic tasks I would say that uncensored models is not a good choice. This is not a problem just for general writing or role-playing, of course.

This is why I like your Gemma-4-26B-A4B-it-Heretic-Cerebellum-GGUF model so much: it is small and fast while still being so good at writing (in my experience Gemma and LLama models have a much better style for generating text than other models). I also saved this model, but with the MTP head grafted in, just in case.

Finally, please do not take my suggestion for Kat-Coder-V2.5 as a request, it was more like a suggestion/question.

Thanks again. Best regards.

deucebucket

Owner 23 days ago

Give it a try. Let me know: https://huggingface.co/deucebucket/KAT-Coder-V2.5-Dev-Cerebellum-GGUF

NeoHuggingF

23 days ago

Downloading now...

I've seen you also did some benchmarks, excellent! Thank you very much for the weight and all the info provided. I will start a discussion there after I run my own tests.

Thanks again. Regards

deucebucket

Owner 23 days ago

Downloading now...

I've seen you also did some benchmarks, excellent! Thank you very much for the weight and all the info provided. I will start a discussion there after I run my own tests.

Thanks again. Regards

No problem! thank you for the thorough testing and suggestions!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment