peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF

Qwen3.5-122B-A10B PS-IQ2_XXS-MTP (Peasant Smith)

The first IQ2_XXS-class GGUF of Qwen3.5-122B-A10B — smaller than any published quant (unsloth's ladder floors at Q3_K_S 52.5 GB) — built entirely from the original Qwen/Qwen3.5-122B-A10B weights with the native MTP (nextn) head embedded at Q8_0.

File: Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf — 41.77 GB (38.9 GiB), ≈2.7 bits/param over 122B total params. SHA-256: 01bd59623e7d4f2f45bd1ce8665042bf449b8000a76480281bef7866cfce5557

Provenance chain (100% original weights)

1. Qwen/Qwen3.5-122B-A10B official BF16 safetensors (39 shards, 244 GB — no third-party quant or modified repo involved)
2. → GGUF master via llama.cpp convert_hf_to_gguf.py (arch qwen35moe, nextn_predict_layers = 1, MTP remapped to blk.48.* and embedded at full precision)
3. → llama-quantize importance-matrix quant with the tensor split below

Imatrix: imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF (359 MB, large diverse calibration corpus). Verified before use: 144/144 main expert tensors covered, zero naming mismatches against our master; the only uncovered tensors are the 3 MTP expert tensors, which were pinned out of IQ quantization.

Tensor split (verified from output file — 0 violations)

Class	Tensors	Type
Routed expert gate/up	96	IQ2_XXS
Routed expert down	48	Q2_K
Token embeddings	1	Q6_K
Attention, shared experts, router, output, norms	384	Q8_0
Layer norms/biases	370	F32 (kept)
MTP / nextn block (blk.48)	20	Q8_0

Down-projection gets one full step above gate/up because its errors land directly and additively on the residual stream, while gate/up errors pass through activations first. The MTP head is kept at maximum fidelity so the draft head stays sharp.

Measured quality checks

Test	Result
Greedy smoke ("The capital of France is")	✅ "The capital of France is Paris." (finish=stop)
Needle recall ("9137 steps", 8k ctx)	✅ exact answer 9137
LRU cache code task (8k ctx)	✅ complete correct doubly-linked-list LRU with sentinel nodes, O(1) get/put, eviction (5000-token continuous generation, zero degradation)
Long-generation stability	✅ 5000 tokens continuous at steady t/s
Perplexity (wikitext-2)	not measured — see honesty note below

Usage

llama-server -m Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf -ngl 99 -ncmoe 20 -c 8192 --load-mode none

Requires a recent llama.cpp master (qwen35moe + linear-attention + MRoPE support). The model ships its chat template; use --jinja.

Limitations

• 2-bit experts: real quality delta vs BF16 exists by construction; task battery above passed but long-tail knowledge degradation is expected at this footprint
• Text-only (vision tower of the original repo is not part of this GGUF)
• Not verified beyond 8k context
• Ollama can load the file, but -ncmoe-style expert pinning is llama.cpp-only

Running with ollama

ollama derives its tag from the GGUF filename (last dash-separated segment):

ollama run hf.co/peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS

Notes:

• The embedded MTP/nextn head is currently inert (llama.cpp has no draft wiring for the qwen35moe architecture yet); the model runs as a standard A10B MoE.
• Expert-offload flags like -ncmoe are llama.cpp-only; ollama loads with its default placement. Measured ~17 t/s with llama.cpp on 3x RTX 3060 (experts partially on CPU) - see benchmarks below.

Credit

Upstream model by Qwen (Apache-2.0). imatrix by Unsloth. Quant by Peasant Smith.