Instructions to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Reinforce-ai/Qwen3.6-35B-A3B-Spiral",
	filename="Qwen3.6-35B-A3B-Spiral_4_5.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral
# Run inference directly in the terminal:
llama cli -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral
# Run inference directly in the terminal:
llama cli -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral
# Run inference directly in the terminal:
./llama-cli -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Use Docker

docker model run hf.co/Reinforce-ai/Qwen3.6-35B-A3B-Spiral

LM Studio
Jan

vLLM

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Reinforce-ai/Qwen3.6-35B-A3B-Spiral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Reinforce-ai/Qwen3.6-35B-A3B-Spiral",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Ollama
How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Ollama:
```
ollama run hf.co/Reinforce-ai/Qwen3.6-35B-A3B-Spiral
```

Unsloth Studio

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Reinforce-ai/Qwen3.6-35B-A3B-Spiral to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Reinforce-ai/Qwen3.6-35B-A3B-Spiral to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Reinforce-ai/Qwen3.6-35B-A3B-Spiral to start chatting

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Reinforce-ai/Qwen3.6-35B-A3B-Spiral"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Reinforce-ai/Qwen3.6-35B-A3B-Spiral" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Docker Model Runner:
```
docker model run hf.co/Reinforce-ai/Qwen3.6-35B-A3B-Spiral
```

Lemonade

How to use Reinforce-ai/Qwen3.6-35B-A3B-Spiral with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Reinforce-ai/Qwen3.6-35B-A3B-Spiral

Run and chat with the model

lemonade run user.Qwen3.6-35B-A3B-Spiral-{{QUANT_TAG}}

List all available models

lemonade list

Qwen3.6-35B-A3B-Spiral / README.md

virajdeshwal

Update README and release slide for v0.3.0

88aeef8 2 months ago

preview code

Raw

History Blame Contribute Delete

9.15 kB

metadata

license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: quantized
language:
  - en
pipeline_tag: text-generation
library_name: spiral
tags:
  - spiral
  - quantization
  - calibration-free
  - gguf
  - llama-cpp
  - qwen
  - qwen3.6
  - moe
  - apple-silicon
  - cuda
  - metal

Qwen3.6-35B-A3B Spiral

Release v0.3.0 — Spiral_4_5 scheme

Calibration-free 5 bpw compression of Qwen3.6-35B-A3B. Competitive with unsloth Q4_K_M on HumanEval (91.5% vs 93.3% base pass@1) at similar artifact size — without calibration data, without fine-tuning, without representative samples. Production-stable on Apple Silicon (Metal) and NVIDIA H100 (CUDA). 91 tok/s decode on H100.

This is the v0.3.0 release of Spiral compression applied to Qwen/Qwen3.6-35B-A3B — the hybrid MoE + DeltaNet architecture from Qwen.

v0.3.0 supersedes v0.2.0 (the prior INT3 release). The new release is a different point in the quality/size Pareto: cleaner inference, parity with Q4-class quants on coding benchmarks, and cross-platform shipping.

What's new vs v0.2.0

Aspect	v0.2.0	v0.3.0 (this release)
Average compression	3.5 bpw	~5.0 bpw
NLL vs bf16	+0.113 nats	~0 nats
HumanEval pass@1	not measured	0.915 base / 0.866 plus
Multi-turn on Mac	broken	works
CUDA support	not shipped	shipped
Calibration data	not required	not required
Production status	research preview	shippable

Quality

HumanEval pass@1

Measured on H100 with llama-server + evalplus, greedy decoding (temp=0), thinking mode disabled:

Method	bpw	Calibration data	HumanEval base	HumanEval+
Spiral_4_5 (this model)	~5.0	None	0.915	0.866
unsloth Q4_K_M (reference)	4.6	Yes (imatrix)	0.933*	0.902*

* Q4_K_M reference numbers from unsloth's community leaderboard submission.

Spiral_4_5 is 1.8 percentage points behind Q4_K_M on base HumanEval, 3.6 points behind on HumanEval+ — and is calibration-free. K-quants require representative data to construct importance matrices for outlier handling; Spiral derives its compression analytically. The model's coding capability is preserved within striking distance of one of the strongest community quantization schemes, without the calibration overhead.

Perplexity (wikitext-2-raw-v1)

Method	bpw	NLL (nats/token)	Gap vs bf16
bf16 (reference)	16.0	1.8968	—
unsloth Q8_0	8.5	1.8962	−0.0006
unsloth Q4_K_XL	5.2	1.8976	+0.0008
Spiral_4_5	~5.0	≈1.8929	≈−0.004

Spiral_4_5 is competitive with Q4_K_XL on perplexity.

Cross-platform validation

The same artifact runs on both Metal (Apple Silicon) and CUDA (NVIDIA H100). Output equivalence has been validated by side-by-side comparison on identical greedy prompts:

Short prompts: byte-identical outputs across both platforms
Long generations: semantically equivalent reasoning, with minor token-level drift from fp16 ordering differences in the deep stack

HumanEval is measured on H100 CUDA. Mac inference produces equivalent quality (validated by cross-platform diff) but has not been benchmarked separately on HumanEval; expect within 1-2 percentage points of the H100 number.

Performance

Apple M2 Max

Configuration	Decode	Prefill
f16 KV + flash attention	~30 tok/s	~80 tok/s

NVIDIA H100 80GB

Configuration	Decode	Prefill
f16 KV + flash attention + CUDA graphs	~91 tok/s	100-220 tok/s

Memory footprint

Qwen3.6-35B-A3B weights + 8K context KV cache:

Method	Total
Spiral_4_5	~20.2 GB
Q4_K_XL	~21.2 GB
Q8_0	~37.2 GB
bf16	~70.2 GB

Fits comfortably in 24 GB of VRAM / unified memory for the 8K-context shipping config.

Files

File	Size
`Qwen3.6-35B-A3B-Spiral_4_5.gguf`	~20 GB
`Qwen3.6-35B-A3B-Spiral_4_5.spiralcb`	28 MB

Both files are required. The .spiralcb sidecar is loaded by the Spiral runtime alongside the GGUF.

Quick start

Spiral ships with three wrappers — spiral-download, spiral-chat, spiral-serve — that handle the model fetch and inference flags.

Install

brew install reinforceai/spiral/spiral

The brew formula installs the Spiral fork of llama.cpp plus the wrappers. Standard upstream llama.cpp will not load Spiral-compressed models.

Interactive chat

spiral-chat --model qwen-36-35b-spiral

First run auto-downloads the GGUF and codebook to ~/.spiral/models/qwen-36-35b-spiral/. Subsequent runs use the local cache.

Single prompt

spiral-chat --model qwen-36-35b-spiral \
    --prompt "Write a Python function to compute Fibonacci numbers iteratively" \
    --greedy

OpenAI-compatible API server

spiral-serve --model qwen-36-35b-spiral --port 8080

curl http://localhost:8080/v1/chat/completions \
    -H 'Content-Type: application/json' \
    -d '{
      "messages": [{"role": "user", "content": "Hello"}],
      "chat_template_kwargs": {"enable_thinking": false}
    }'

enable_thinking: false disables Qwen's thinking-block emission — matches how the HumanEval scores in this README were measured. Omit it for full thinking traces.

Build from source

For CUDA, or to develop against the framework:

git clone https://github.com/ReinforceAI/spiral
cd spiral
cmake -B build -DGGML_METAL=ON   # or -DGGML_CUDA=ON
cmake --build build -j

Then spiral-download and spiral-chat from the build directory, or call llama-cli directly with SPIRAL_CODEBOOK_PATH set to the codebook file.

How it works

Spiral applies a dense rotation to weight matrices before scalar quantization. The rotation produces near-Gaussian weight distributions with equalized variance, eliminating outlier channels — the primary source of quantization error in coder models. Quantization in the rotated space is then much more efficient than in the original basis.

Spiral_4_5 uses mixed bit depth across MoE expert tensors, with higher precision allocated to a subset of layers identified through per-layer sensitivity analysis. Attention and embedding weights remain in higher precision.

No calibration data is required. The compression is derived analytically from the model's own weights — no representative samples, no importance matrix construction, no fine-tuning.

Full method description is in development. Source: github.com/ReinforceAI/spiral

Limitations

KV cache compression not in this release. Long-context use cases that would benefit from KV compression are not addressed by v0.3.0; future releases will revisit.
HumanEval measured on H100 only. Mac HumanEval is expected within 1-2pp based on cross-platform diff validation.
Custom llama.cpp build required. Stock upstream llama.cpp does not load Spiral models.
Coding benchmarks only. HumanEval is the primary evaluation. MBPP, LiveCodeBench, SWE-bench have not been measured in this release.
Research-grade release. APIs may change before 1.0.

Methodology

HumanEval

evalplus 0.3.1, llama-server on H100 80GB, CUDA backend
Greedy decoding (temperature=0)
Thinking mode disabled (enable_thinking: false)
164 problems, single sample per problem

Perplexity

wikitext-2-raw-v1, HF AutoTokenizer
64 sequences × 2048 tokens = 131,008 tokens
Cross-method protocol: same tokenizer source across all comparisons, no GGUF tokenizer invocation for K-quants (avoids tokenizer mismatch as a confound)

Cross-platform validation

Side-by-side llama-cli with identical greedy prompts on Mac (M2 Max, Metal) and H100 (CUDA)
Same .gguf + .spiralcb on both platforms

Reproduction scripts: github.com/ReinforceAI/spiral

Acknowledgments

llama.cpp — inference engine, GGUF format, Metal and CUDA backends.
unsloth — high-quality GGUF quantizations of Qwen3.6-35B-A3B and many other models, used as the reference comparison point in this release (Q4_K_M, Q4_K_XL, Q8_0).
Qwen Team — Qwen3.6-35B-A3B under Apache 2.0.
The broader open-source ML community — quantization theory, rotation methods, and product quantization research laid the groundwork.

Citation

@misc{spiral2026,
  title={Spiral: Geometric Compression of Rotated Transformers},
  author={Deshwal, Viraj},
  year={2026},
  publisher={ReinforceAI},
  url={https://github.com/ReinforceAI/spiral}
}

License

Inference engine: Based on llama.cpp (MIT)
Spiral compression framework: ReinforceAI
Model weights: Apache 2.0 (inherited from Qwen3.6-35B-A3B)