Instructions to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
# Run inference directly in the terminal:
llama cli -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
# Run inference directly in the terminal:
llama cli -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
# Run inference directly in the terminal:
./llama-cli -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Use Docker

docker model run hf.co/ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

LM Studio
Jan

vLLM

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Ollama
How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Ollama:
```
ollama run hf.co/ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
```

Unsloth Studio

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF to start chatting

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Docker Model Runner:
```
docker model run hf.co/ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF
```

Lemonade

How to use ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-heretic-750-v1-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Experimental global target bits‑per‑weight quantization of tvall43/Qwen3.6-35B-A3B-heretic

Using non-standard (forked) LLaMA C++ branch for quantization.
Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
Using dataset sources: tools, math, code, text_en, text_ru.
Using dataset chunks: 750.
Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
Small set of patches added.
Multimodal work perfectly, use:

Many thanks to Ed Addario for an impressive job.

Quantization comparison

BPW/TGS	SIZE (MiB)	PPL correlation	PPL mean ratio	ΔPPL	Mean KLD	Median KLD	Maximum KLD	99.9% KLD	Mean Δp	RMS Δp
2.50	0	93.70%	1.322174 ± 0.002834	2.368983 ± 0.024164	0.331479 ± 0.001228	0.199301	22.199869	6.503164	-5.508 ± 0.036 %	17.170 ± 0.059 %
2.75	0	94.44%	1.269795 ± 0.002563	1.983838 ± 0.021645	0.283522 ± 0.001164	0.166356	21.203295	6.781775	-4.261 ± 0.034 %	15.785 ± 0.059 %
3.00	0	96.17%	1.179408 ± 0.001950	1.319209 ± 0.015874	0.202655 ± 0.000892	0.122186	19.826981	5.150637	-3.425 ± 0.029 %	13.351 ± 0.055 %
3.25	0	96.59%	1.153705 ± 0.001791	1.130213 ± 0.014295	0.180917 ± 0.000807	0.111134	18.365435	5.041100	-3.155 ± 0.027 %	12.615 ± 0.052 %
3.50	0	97.67%	1.091825 ± 0.001399	0.675203 ± 0.010821	0.123820 ± 0.000686	0.074732	22.352665	4.073599	-1.971 ± 0.022 %	10.253 ± 0.048 %
3.75	0	98.67%	1.043301 ± 0.001000	0.318400 ± 0.007366	0.065548 ± 0.000531	0.033836	23.969786	2.976686	-1.584 ± 0.017 %	7.681 ± 0.049 %
4.00	0	98.81%	1.030391 ± 0.000934	0.223470 ± 0.006807	0.058529 ± 0.000469	0.031020	14.197460	2.886632	-1.363 ± 0.016 %	7.234 ± 0.048 %
4.25	0	98.86%	1.025584 ± 0.000912	0.188119 ± 0.006637	0.055969 ± 0.000488	0.029261	15.963261	2.994938	-1.252 ± 0.015 %	7.060 ± 0.049 %
4.50	0	98.98%	1.029716 ± 0.000870	0.218506 ± 0.006505	0.048650 ± 0.000511	0.022877	18.484322	2.939611	-0.376 ± 0.014 %	6.499 ± 0.052 %
4.75	0	99.33%	1.015859 ± 0.000694	0.116613 ± 0.005141	0.028482 ± 0.000415	0.011463	17.854448	2.228527	-0.140 ± 0.012 %	5.200 ± 0.056 %
5.00	0	99.36%	1.012113 ± 0.000676	0.089068 ± 0.004982	0.026222 ± 0.000441	0.010419	18.202673	2.060771	-0.128 ± 0.011 %	4.899 ± 0.056 %
5.25	0	99.44%	1.012173 ± 0.000630	0.089508 ± 0.004653	0.023537 ± 0.000371	0.009648	17.899471	1.706400	-0.146 ± 0.010 %	4.639 ± 0.052 %
5.50	0	99.47%	1.011954 ± 0.000618	0.087896 ± 0.004568	0.022571 ± 0.000377	0.008998	17.762650	1.721754	-0.138 ± 0.010 %	4.592 ± 0.054 %
5.75	0	99.48%	1.008779 ± 0.000607	0.064554 ± 0.004470	0.021041 ± 0.000368	0.008366	16.971138	1.446258	-0.144 ± 0.010 %	4.391 ± 0.052 %
6.00	0	99.68%	1.004470 ± 0.000475	0.032865 ± 0.003498	0.012901 ± 0.000301	0.004279	17.765589	1.038293	0.032 ± 0.008 %	3.429 ± 0.051 %
6.25	0	99.66%	1.006531 ± 0.000492	0.048024 ± 0.003629	0.014493 ± 0.000336	0.005697	17.488436	1.112754	-0.009 ± 0.008 %	3.719 ± 0.052 %
6.50	0	99.70%	1.003215 ± 0.000460	0.023639 ± 0.003385	0.013321 ± 0.000314	0.005327	17.561293	0.926146	-0.027 ± 0.008 %	3.557 ± 0.050 %
6.75	0	99.72%	1.002924 ± 0.000442	0.021502 ± 0.003248	0.012513 ± 0.000323	0.004834	18.359007	0.841218	-0.034 ± 0.008 %	3.403 ± 0.049 %
7.00	0	99.72%	1.003523 ± 0.000447	0.025909 ± 0.003289	0.011451 ± 0.000291	0.004472	16.875040	0.810624	-0.010 ± 0.007 %	3.333 ± 0.051 %
7.25	0	99.73%	1.005361 ± 0.000433	0.039421 ± 0.003188	0.010976 ± 0.000318	0.003979	22.955790	0.796251	-0.065 ± 0.007 %	3.194 ± 0.049 %
7.50	0	99.75%	1.004073 ± 0.000417	0.029952 ± 0.003068	0.010470 ± 0.000297	0.003846	17.177761	0.723907	-0.046 ± 0.007 %	3.172 ± 0.050 %
7.75	0	99.76%	1.003395 ± 0.000413	0.024961 ± 0.003037	0.010345 ± 0.000295	0.003763	16.866329	0.780084	-0.043 ± 0.007 %	3.190 ± 0.052 %
8.00	33065	99.76%	1.003632 ± 0.000410	0.026709 ± 0.003015	0.010329 ± 0.000297	0.003678	16.988951	0.787349	-0.044 ± 0.007 %	3.186 ± 0.053 %
8.25	34098	99.77%	1.003770 ± 0.000400	0.027723 ± 0.002942	0.010028 ± 0.000278	0.003677	16.766968	0.706373	-0.044 ± 0.007 %	3.146 ± 0.051 %
8.50	35131	99.78%	1.003254 ± 0.000396	0.023930 ± 0.002915	0.009253 ± 0.000283	0.003388	23.840731	0.693518	-0.040 ± 0.007 %	2.998 ± 0.050 %
8.75	36164	99.78%	1.001925 ± 0.000389	0.014155 ± 0.002859	0.009190 ± 0.000301	0.003274	23.258368	0.654125	-0.041 ± 0.007 %	3.000 ± 0.051 %
9.00	37197	99.78%	1.003080 ± 0.000396	0.022649 ± 0.002917	0.009205 ± 0.000282	0.003272	17.987577	0.659261	-0.035 ± 0.007 %	2.993 ± 0.051 %
9.25	38230	99.77%	1.002496 ± 0.000397	0.018354 ± 0.002920	0.009125 ± 0.000298	0.003269	23.666058	0.616762	-0.043 ± 0.007 %	2.960 ± 0.050 %
9.50	39263	99.78%	1.002410 ± 0.000396	0.017724 ± 0.002912	0.009165 ± 0.000283	0.003284	15.693835	0.665747	-0.038 ± 0.007 %	2.996 ± 0.051 %
9.75	40296	99.79%	1.002872 ± 0.000388	0.021119 ± 0.002852	0.009108 ± 0.000286	0.003272	16.791821	0.625508	-0.029 ± 0.007 %	2.981 ± 0.052 %
10.00	41329	99.77%	1.002888 ± 0.000399	0.021235 ± 0.002935	0.008968 ± 0.000260	0.003268	15.437628	0.698991	-0.042 ± 0.007 %	2.960 ± 0.050 %
10.25	42362	99.79%	1.002763 ± 0.000385	0.020317 ± 0.002832	0.008947 ± 0.000267	0.003271	17.035978	0.679305	-0.041 ± 0.007 %	2.955 ± 0.049 %
10.50	43395	99.78%	1.002359 ± 0.000392	0.017345 ± 0.002883	0.009061 ± 0.000278	0.003255	16.101616	0.675837	-0.044 ± 0.007 %	2.998 ± 0.052 %
10.75	44428	99.78%	1.003859 ± 0.000392	0.028373 ± 0.002885	0.008878 ± 0.000260	0.003267	16.314154	0.642667	-0.038 ± 0.007 %	2.952 ± 0.049 %
11.00	45460	99.78%	1.002777 ± 0.000390	0.020422 ± 0.002869	0.008851 ± 0.000271	0.003232	15.418838	0.575241	-0.044 ± 0.006 %	2.903 ± 0.048 %
11.25	46493	99.78%	1.002854 ± 0.000389	0.020985 ± 0.002858	0.009240 ± 0.000318	0.003199	22.994694	0.666638	-0.032 ± 0.007 %	2.977 ± 0.052 %
11.50	47526	99.78%	1.002975 ± 0.000393	0.021874 ± 0.002893	0.008938 ± 0.000279	0.003185	15.113097	0.631946	-0.037 ± 0.007 %	2.959 ± 0.052 %
11.75	48559	99.79%	1.003570 ± 0.000380	0.026254 ± 0.002796	0.009098 ± 0.000306	0.003182	23.414881	0.686298	-0.040 ± 0.007 %	3.000 ± 0.053 %
12.00	49592	99.79%	1.003774 ± 0.000386	0.027747 ± 0.002842	0.009182 ± 0.000310	0.003198	23.352367	0.643423	-0.051 ± 0.007 %	2.986 ± 0.054 %
12.25	50625	99.78%	1.002765 ± 0.000392	0.020330 ± 0.002881	0.009154 ± 0.000298	0.003182	17.150059	0.641994	-0.029 ± 0.007 %	3.003 ± 0.053 %
12.50	51658	99.78%	1.002484 ± 0.000396	0.018269 ± 0.002915	0.009119 ± 0.000319	0.003167	23.075504	0.621232	-0.039 ± 0.007 %	2.977 ± 0.052 %
12.75	52691	99.79%	1.003419 ± 0.000387	0.025141 ± 0.002850	0.009195 ± 0.000308	0.003181	23.022285	0.658149	-0.043 ± 0.007 %	2.978 ± 0.052 %
13.00	53723	99.79%	1.002973 ± 0.000383	0.021863 ± 0.002817	0.009105 ± 0.000290	0.003179	22.760689	0.717768	-0.041 ± 0.007 %	2.984 ± 0.053 %
13.25	54757	99.79%	1.002846 ± 0.000383	0.020924 ± 0.002817	0.009158 ± 0.000293	0.003157	17.122913	0.643450	-0.037 ± 0.007 %	2.973 ± 0.051 %
13.50	55790	99.78%	1.003739 ± 0.000391	0.027492 ± 0.002881	0.008980 ± 0.000292	0.003187	23.291426	0.602581	-0.046 ± 0.007 %	2.964 ± 0.052 %
13.75	56820	99.80%	1.002970 ± 0.000375	0.021841 ± 0.002756	0.009048 ± 0.000295	0.003163	14.807920	0.574495	-0.044 ± 0.007 %	2.952 ± 0.051 %
14.00	57855	99.78%	1.003342 ± 0.000390	0.024571 ± 0.002871	0.008916 ± 0.000296	0.003160	23.040359	0.598707	-0.040 ± 0.007 %	2.957 ± 0.052 %
14.25	58889	99.80%	1.003795 ± 0.000377	0.027908 ± 0.002774	0.009072 ± 0.000293	0.003174	23.050707	0.739776	-0.047 ± 0.007 %	3.022 ± 0.053 %
14.50	59922	99.79%	1.003739 ± 0.000386	0.027496 ± 0.002841	0.009037 ± 0.000300	0.003159	23.497532	0.689851	-0.051 ± 0.007 %	2.966 ± 0.053 %
14.75	60953	99.78%	1.002725 ± 0.000389	0.020035 ± 0.002860	0.009020 ± 0.000301	0.003157	23.477308	0.675409	-0.033 ± 0.007 %	2.990 ± 0.053 %
15.00	61988	99.78%	1.003097 ± 0.000391	0.022776 ± 0.002875	0.009012 ± 0.000308	0.003166	23.145992	0.627263	-0.038 ± 0.007 %	2.987 ± 0.055 %
15.25	63021	99.79%	1.003312 ± 0.000387	0.024355 ± 0.002843	0.009065 ± 0.000295	0.003179	23.164423	0.701907	-0.048 ± 0.007 %	2.985 ± 0.052 %
15.50	64050	99.79%	1.002819 ± 0.000380	0.020730 ± 0.002796	0.008938 ± 0.000301	0.003166	23.532108	0.607342	-0.038 ± 0.007 %	2.923 ± 0.051 %
15.75	65085	99.78%	1.002776 ± 0.000391	0.020414 ± 0.002873	0.008980 ± 0.000302	0.003156	23.250616	0.657160	-0.031 ± 0.007 %	2.994 ± 0.054 %

Downloads last month: 310

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

tvall43/Qwen3.6-35B-A3B-heretic

Quantized

(15)

this model

ENOSYS
/

Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Experimental global target bits‑per‑weight quantization of tvall43/Qwen3.6-35B-A3B-heretic

Quantization comparison

Model tree for ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF

Dataset used to train ENOSYS/Qwen3.6-35B-A3B-heretic-750-v1-GGUF