Instructions to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Use Docker

docker model run hf.co/anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Ollama
How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Ollama:
```
ollama run hf.co/anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
```

Unsloth Studio

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF to start chatting

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Docker Model Runner:
```
docker model run hf.co/anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M
```

Lemonade

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-coding-reap25-healed-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.6-35B-A3B coding specialist, 25% experts kept, healed (GGUF)

Companion checkpoint for Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding (arXiv:2607.16721). Pipeline and the exact keep-set JSON that produced it: github.com/anik-jha/moep.

75% of the routed experts were removed from Qwen3.6-35B-A3B under the REAP criterion calibrated on a coding-weighted mix, then the survivors were healed by distilling from the base model with the expert tensors frozen (LoRA on attention, router, and shared expert only).

Read this before using it

This is a research artifact, and the paper's headline finding is a negative one: at this memory budget you are better off quantizing the full model than pruning it. The healed checkpoint below sits at 11.2 GB and scores 0.841 on HumanEval+. A 2-bit IQ2_M quantization of the unpruned model sits at 12.1 GB and scores 0.896. Pruning to 25% and healing does not close that gap; five separate attempts to close it are reported in the paper and all failed.

The checkpoint is released because healing is GPU-nondeterministic and therefore not reproducible from the keep-set JSON, unlike every other point in the study. It is the artifact behind the paper's healing result, not a recommended deployment. If you want something to actually run, use the 50%-keep specialist: Qwen3.6-35B-A3B-coding-reap50-GGUF.

Evaluations

All numbers from runs/qwen36/eval.json in the pipeline repo, greedy decoding, EvalPlus HumanEval+ and MBPP+ (pass@1). Only the Q8_0 file was functionally evaluated.

Checkpoint	Size	HumanEval+	MBPP+
Base Qwen3.6-35B-A3B, Q8_0	36 GB	0.890	0.772
Base, full model, IQ2_M (iso-memory comparator)	12.1 GB	0.896	0.730
25% keep, REAP, no healing, Q8_0	11.2 GB	0.787	0.659
25% keep, REAP, healed, Q8_0 (this repo)	11.2 GB	0.841	0.685

Healing recovers 54 of the 103 HumanEval+ points lost to pruning (0.787 → 0.841 against a base of 0.890) and 26 of 113 on MBPP+. Held-out perplexity improves from 5.45 to 4.85 on code and from 30.9 to 9.4 on general text.

The Q4_K_M file in this repo has not been functionally evaluated. It passed an export smoke test and was benchmarked for throughput, nothing more. Do not attribute the table above to it.

Files

File	Size	Notes
`qwen36-reap25-healed-Q8_0.gguf`	11.2 GB	evaluated, numbers above
`qwen36-reap25-healed-Q4_K_M.gguf`	6.5 GB	smoke-tested only, not evaluated

Throughput measured with llama-bench on one DGX Spark (GB10, 119 GB unified memory): Q8_0 60.7 tok/s generation, 2780 tok/s prefill; Q4_K_M 82.0 tok/s generation, 3081 tok/s prefill.

License

Weights are a derivative of Qwen3.6-35B-A3B and inherit its Apache-2.0 license. The pipeline that produced them is MIT.

Downloads last month: 572

GGUF

Model size

10B params

Architecture

qwen35moe

Hardware compatibility

4-bit

8-bit

Model tree for anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Quantized

(751)

this model

Paper for anik-jha/Qwen3.6-35B-A3B-coding-reap25-healed-GGUF

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

Paper • 2607.16721 • Published about 1 month ago