Instructions to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Use Docker

docker model run hf.co/Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

LM Studio
Jan

vLLM

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Ollama
How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Ollama:
```
ollama run hf.co/Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
```

Unsloth Studio

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF to start chatting

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Docker Model Runner:
```
docker model run hf.co/Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16
```

Lemonade

How to use Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Jianqiao1/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF:BF16

Run and chat with the model

lemonade run user.Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF-BF16

List all available models

lemonade list

Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MoQ-GGUF / README.md

Jianqiao1

Upload README.md with huggingface_hub

95999d8 verified 4 days ago

7.41 kB

license: apache-2.0
base_model: nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451
base_model_relation: quantized
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - qwen3.6
  - quantized
  - moq
  - mtp
  - nvfp4

Qwen3.6-27B Architect Polaris2 Fable B F451 - MTP MoQ GGUF

Tensor-aware mixed-precision GGUF quantizations of nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451, including the model's MTP tensors.

This repository provides several MoQ size/quality points, an experimental NVFP4 variant, the c128 importance matrix used during quantization, and the complete interactive evaluation report.

Quantization method

MoQ uses tensor-aware mixed quantization instead of assigning one quantization type to the entire model.

Weight tensors are grouped by their structural role and sensitivity.
Per-group KLD sensitivity studies are used to decide which groups require higher precision.
Less sensitive groups use more compact formats, while sensitive groups are protected with higher-precision formats.
A 128-chunk importance matrix is incorporated into the quantization process.
The large MTP tensors are explicitly retained at high precision.
Every output was audited against its tensor recipe: all 866 tensor names, shapes, and quantization types matched.

The MTP-MoQ-NVFP4 variant uses NVFP4 for selected tensor groups while retaining the rest of the mixed-precision policy. On an RTX 5090 it improves short-prompt prefill throughput substantially, although the regular MoQ-4.8 model provides better KLD quality at the same file size.

No single quant is best for every use case:

MTP-MoQ-4.6 is a compact balanced option.
MTP-MoQ-4.8 is the recommended quality/size point around 16.15 GB.
MTP-MoQ-5.1 provides the best quality in the regular MoQ series.
MTP-MoQ-NVFP4 is intended for testing NVFP4 acceleration, especially on Blackwell GPUs.

Files

Sizes are decimal GB. Payload BPW is calculated from the actual GGUF tensor payload and excludes the GGUF header.

File	Size GB	Payload BPW	PPL	Mean KLD	p999 KLD
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.2.gguf`	10.810	3.1621	6.805998	0.097714	2.751078
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.6.gguf`	12.134	3.5500	6.661694	0.069665	1.796203
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.8.gguf`	12.840	3.7566	6.518863	0.044993	1.291825
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.1.gguf`	14.266	4.1740	6.447097	0.027318	0.807045
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.3.gguf`	15.002	4.3897	6.421027	0.019253	0.591232
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.6.gguf`	15.242	4.4599	6.392239	0.015145	0.505663
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.8.gguf`	16.150	4.7258	6.379456	0.012836	0.423877
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.9.gguf`	16.524	4.8353	6.384560	0.012218	0.402534
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-5.1.gguf`	17.452	5.1069	6.371239	0.009613	0.325677
`Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-NVFP4.gguf`	16.150	4.7258	6.397915	0.015615	0.518097

Additional files:

Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-BF16.gguf: BF16 GGUF reference model.
Qwen3.6-27B-Architect-Polaris2-Fable-B-F451.c128.imatrix.gguf: the 128-chunk importance matrix used for these quantizations.
Jianqiao1_MoQ_vs_DavidAU_GGUF_report.html: self-contained interactive report with zoomable and pannable charts.

Quality evaluation

All quality results were measured under the same conditions:

WikiText-2 raw test set
Context length 512
580 evaluation chunks
KLD calculated against logits from the BF16 model
BF16 reference PPL: 6.320035

Lower PPL, Mean KLD, and p999 KLD are better.

MoQ results

Model	Size GB	Payload BPW	PPL	Mean KLD	p999 KLD
MTP-MoQ-3.2	10.810	3.1621	6.805998	0.097714	2.751078
MTP-MoQ-3.6	12.134	3.5500	6.661694	0.069665	1.796203
MTP-MoQ-3.8	12.840	3.7566	6.518863	0.044993	1.291825
MTP-MoQ-4.1	14.266	4.1740	6.447097	0.027318	0.807045
MTP-MoQ-4.3	15.002	4.3897	6.421027	0.019253	0.591232
MTP-MoQ-4.6	15.242	4.4599	6.392239	0.015145	0.505663
MTP-MoQ-4.8	16.150	4.7258	6.379456	0.012836	0.423877
MTP-MoQ-4.9	16.524	4.8353	6.384560	0.012218	0.402534
MTP-MoQ-5.1	17.452	5.1069	6.371239	0.009613	0.325677
MTP-MoQ-NVFP4	16.150	4.7258	6.397915	0.015615	0.518097

Comparison with DavidAU Fable Fusion 711 GGUF

The following DavidAU GGUF files were evaluated locally using the same BF16 reference logits and the same evaluation settings.

DavidAU quant	Size GB	Payload BPW	PPL	Mean KLD	p999 KLD
IQ2_M	12.125	3.5471	7.100246	0.140480	3.807066
IQ3_M	14.532	4.2520	6.545219	0.047823	1.487676
IQ4_XS	17.034	4.9846	6.387851	0.013953	0.514973
Q4_K_S	17.537	5.1321	6.378748	0.013965	0.492694
IQ4_NL	17.753	5.1952	6.386419	0.013701	0.492569
Q4_K_M	18.499	5.4135	6.365021	0.011404	0.412386
Q5_K_S	20.631	6.0379	6.341159	0.005466	0.211335
Q5_K_M	21.182	6.1993	6.337103	0.004809	0.179674
Q6_K	24.034	7.0343	6.324763	0.001467	0.059772

Near-size comparison

Positive percentages mean the MoQ model is smaller or has a lower metric.

MoQ model	DavidAU model	Size difference	Mean KLD lower	p999 KLD lower	PPL lower
MTP-MoQ-3.6	IQ2_M	-0.08%	50.41%	52.82%	6.18%
MTP-MoQ-4.1	IQ3_M	1.83%	42.88%	45.75%	1.50%
MTP-MoQ-4.8	IQ4_XS	5.19%	8.01%	17.69%	0.13%
MTP-MoQ-5.1	Q4_K_S	0.49%	31.16%	33.90%	0.12%

Quality charts

Model size vs p999 KLD

Model size vs Mean KLD

Model size vs WikiText-2 PPL

For interactive zooming, panning, series selection, and detailed hover values, download and open Jianqiao1_MoQ_vs_DavidAU_GGUF_report.html.

RTX 5090 performance

Throughput was measured with f16 KV cache and three repetitions. MTP-MoQ-NVFP4 and MTP-MoQ-4.8 have exactly the same 16.150 GB file size.

Model	pp512 tok/s	tg128 tok/s	pg32768,256 tok/s
MTP-MoQ-NVFP4	3256.97	78.92	2377.72
MTP-MoQ-4.8	2669.45	78.65	2356.68
MTP-MoQ-4.6	2731.86	82.87	2400.43
NVFP4 vs MoQ-4.8	+22.01%	+0.34%	+0.89%

License

This repository is released under the Apache License 2.0. Users should also review and comply with the terms of the original model.

Acknowledgements

Thanks to nightmedia for the original Qwen3.6-27B Architect Polaris2 Fable B F451 model.
Thanks to DavidAU for the Fable Fusion 711 GGUF series used in the comparison.
Thanks to the Qwen team for Qwen3.6.