release: Qwen3.6-35B-A3B Spiral 3-bit

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
+*.spiralcb filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,162 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.6-35B-A3B
+base_model_relation: quantized
+language:
+  - en
+pipeline_tag: text-generation
+library_name: spiral
+tags:
+  - spiral
+  - quantization
+  - int3
+  - kv-cache-compression
+  - apple-silicon
+  - metal
+  - gguf
+  - llama-cpp
+  - qwen
+  - qwen3.6
+  - moe
+  - deltanet
+---
+
+# Qwen3.6-35B-A3B Spiral
+
+**Geometric compression of Qwen3.6-35B-A3B.** 3.5 bits per weight. 7.1× K-cache compression. +0.113 nats vs bf16 on wikitext-2. Runs at 23–24 tok/s on Apple M2 Max.
+
+This is a Spiral-compressed version of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) — the hybrid MoE + DeltaNet architecture from Qwen, compressed via Walsh-Hadamard rotation and Lloyd-Max INT3 weights with product-quantized K cache.
+
+![Qwen3.6-35B-A3B Spiral release results](spiral_release_slide.svg)
+
+## Quality
+
+Cross-method perplexity comparison on H100, identical methodology (HF AutoTokenizer, 131,008 wikitext-2 tokens, same NLL formula across all methods):
+
+| Method | Bits/weight | Disk size | NLL (nats/token) | Gap vs bf16 |
+|---|---|---|---|---|
+| bf16 (reference) | 16.0 | 67.0 GB | 1.8968 | — |
+| unsloth Q8_0 | 8.5 | 34.4 GB | 1.8962 | −0.0006 |
+| unsloth Q4_K_XL | 5.2 | 20.8 GB | 1.8976 | +0.0008 |
+| **Spiral 3-bit (this model)** | **3.5** | **14.2 GB** | **2.0101** | **+0.1133** |
+
+Spiral is **1.46× smaller than Q4_K_XL** for +0.11 nats of additional perplexity — practically invisible in generation.
+
+## KV cache compression
+
+Spiral additionally compresses the K cache via product quantization (256 codewords × 4-dim subspaces). Unlike weight quantization, KV compression is online — every token at inference time encodes through PQ on write and decodes on read.
+
+| Method | bytes / token | K compression | Combined K+V |
+|---|---|---|---|
+| f16 (default for Q8_0, Q4_K_XL, bf16) | 20,480 | 1× | 1× |
+| **Spiral PQ K + f16 V** | **11,680** | **7.1×** | **1.75×** |
+
+At 1M context, this saves 8.6 GiB — the difference between fitting on a 64 GB Mac or not.
+
+## Total memory at 1M context
+
+Weights + KV cache for Qwen3.6-35B-A3B:
+
+| Method | Total at 1M | Fits 96 GB Mac? | Fits 64 GB Mac? |
+|---|---|---|---|
+| **Spiral 3-bit** | **25.6 GiB** | ✓ comfortable | ✓ comfortable |
+| Q4_K_XL | 40.8 GiB | ✓ | tight |
+| Q8_0 | 54.4 GiB | tight | ✗ |
+| bf16 | 87.0 GiB | ✗ | ✗ |
+
+Add ~1 GiB compute scratch for realistic peak.
+
+## Files in this repo
+
+| File | Size | Purpose |
+|---|---|---|
+| `qwen36-35b-a3b.gguf` | 14.0 GB | Quantized weights (INT3 + rotation) |
+| `qwen36-35b-a3b-codebooks.spiralcb` | 11 MB | PQ codebooks + rotation seeds |
+| `README.md` | — | This model card |
+| `spiral_release_slide.svg` | 14 KB | Results visualization |
+
+**Both the GGUF and the `.spiralcb` file are required.** The codebooks contain the Walsh-Hadamard rotation parameters and PQ codebook centroids — they are loaded at inference time alongside the weights.
+
+## Quick start (Apple Silicon)
+
+Spiral runs via a custom build of `llama.cpp` with Metal kernels for fused PQ flash attention.
+
+### Install
+
+```bash
+brew install reinforceai/spiral/spiral
+```
+
+### Run
+
+```bash
+spiral-chat --model qwen-36-35b-spiral
+```
+
+The first run downloads this repo's files. Subsequent runs use the local cache.
+
+### Manual download
+
+```bash
+hf download Reinforce-ai/Qwen3.6-35B-A3B-Spiral \
+    qwen36-35b-a3b.gguf qwen36-35b-a3b-codebooks.spiralcb \
+    --local-dir ./models/spiral-35b/
+```
+
+Then point `spiral-chat` at the local directory.
+
+## Performance (Apple M2 Max)
+
+| Configuration | Decode | Prefill |
+|---|---|---|
+| PQ KV + flash attention | 23–24 tok/s | 80 tok/s |
+| f16 KV | 36 tok/s | — |
+
+PQ KV trades ~30% decode speed for 7.1× K compression. For long-horizon agent tasks where context capacity is the binding constraint, the trade is favorable. Switch with `-ctk f16` at the CLI.
+
+## How it works
+
+Spiral applies multi-pass block Walsh-Hadamard rotation to weights before INT3 quantization. The rotation makes weight distributions near-Gaussian with equalized variance across all dimensions — outlier channels (the primary source of quantization error) vanish under rotation. Lloyd-Max scalar quantization in the rotated space is then optimal.
+
+For the K cache, product quantization with learned codebooks compresses each 4-dim subspace independently into 8-bit codes. A fused Metal kernel decodes PQ codes, applies RoPE, and computes attention in a single pass — no intermediate decompressed tensor is materialized.
+
+For the hybrid Qwen3.6-35B-A3B architecture (10 full attention layers + 30 DeltaNet linear attention layers, 256 MoE experts with 8 active per token), Spiral applies weight compression uniformly across all layer types, with KV compression specific to the 10 attention layers (DeltaNet has no KV cache).
+
+Full method documentation, source, and reproduction scripts: https://github.com/ReinforceAI/spiral
+
+## Methodology
+
+The +0.1133 nats gap is measured with rigorous cross-method protocol on H100:
+
+- HuggingFace AutoTokenizer applied to wikitext-2-raw-v1 validation, **single tokenizer source of truth** for all four methods
+- 64 sequences × 2048 tokens = 131,008 tokens
+- bf16 and Spiral evaluated via PyTorch `F.cross_entropy`
+- Q8_0 and Q4_K_XL evaluated via `llama-cpp-python` with **pre-tokenized inputs** (logits_all=True) — the GGUF tokenizer is never invoked, eliminating tokenizer mismatch as a confound
+- All numbers from the same H100 pod, same eval session
+
+The eval script `eval_cross_method.py` is included in the [GitHub repo](https://github.com/ReinforceAI/spiral) for independent reproduction.
+
+## Limitations
+
+- Apple Silicon focus. CUDA kernels not yet shipped — Spiral models on Linux/CUDA require fp16 fallback for inference. Mac is the production target.
+- PQ KV cache is currently K-only (V stays at fp16). Full K+V PQ is in development; would extend the 1.75× combined compression toward 7.1×.
+- Brew formula installs a custom `llama.cpp` build with Spiral kernels. Standard llama.cpp builds will not load the SPIRAL_3BIT or SPIRAL_PQ2 GGUF types.
+- This is a research preview. APIs may change before 1.0.
+
+## Citation
+
+```bibtex
+@misc{spiral2026,
+  title={Spiral: Geometric Compression of Rotated Transformers},
+  author={Deshwal, Viraj},
+  year={2026},
+  publisher={ReinforceAI},
+  url={https://github.com/ReinforceAI/spiral}
+}
+```
+
+## License
+
+Inference engine: Based on llama.cpp (MIT)
+Spiral compression framework: ReinforceAI
+Model weights: Apache 2.0 (inherited from Qwen3.6-35B-A3B)

qwen36-35b-a3b-codebooks.spiralcb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b90f087b69b73905798ae0ee0ad55e474d6bab6c69da5908b96b383e1c123bb
+size 11190440

qwen36-35b-a3b.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a56745857654215b8bbc30afd64d7fde6d28727932d8c1f9159b9ca5a978b6b
+size 15281584160