---
datasets:
- Glint-Research/Fable-5-traces
- angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k
- osieosie/tmax-sft-skill-tax-20260505-2.2k-combined-balanced-qwen3.6-27b-thinking
- zake7749/Qwen3.6-35B-A3B-Tool-Calling
- nickrosh/Evol-Instruct-Code-80k-v1
tags:
- gguf
- qwen
- llama.cpp
- text-generation
- vision
license: apache-2.0
base_model: tvall43/Qwen3.6-14B-A3B-FableVibes
language: en
---
# Qwen3.6-14B-A3B-FableVibes-GGUF

GGUF quantizations of [Qwen3.6-14B-A3B-FableVibes](https://huggingface.co/tvall43/Qwen3.6-14B-A3B-FableVibes), a 14B MoE model fine-tuned on reasoning traces from Claude Fable 5.

## Background

This model started as `Qwen3.6-35B-A3B-heretic` and was pruned via REAP down to ~14B active parameters, removing over half its expert capacity. A single QLoRA pass was then orchestrated entirely by an autonomous AI agent (**Steve**), utilizing ~4,600 raw reasoning traces from Claude Fable 5 (Mythos-class) to recover capabilities lost during pruning. 

Rather than focusing strictly on agentic orchestration, this model serves as a general-purpose reasoning distill. The Fable CoT traces provide structured multi-step reasoning patterns from a frontier-class model, distilled into a footprint that can run on consumer hardware. The Fable traces are further supplemented by Claude Opus reasoning, Qwen tool-calling data, and Evol-Instruct-Code.

## Available Formats

| Quant | Size | Notes |
|-------|------|-------|
| F16 | ~27GB | Full precision reference |
| Q8_0 | ~15GB | Near-lossless |
| Q6_K | ~11.3GB | Quality/size sweet spot |
| Q5_K_M | ~9.8GB | High quality |
| BPW4.75 | ~8.5GB | Custom exl2-matched quantization array |
| Q4_K_M | ~8.4GB | Recommended for 8-12GB VRAM |
| Q3_K_M | ~6.7GB | Tight fits |
| Q2_K | ~5.3GB | Maximum compression |

Vision support (mmproj files) is included for multimodal use.

## Usage

Works with any llama.cpp-compatible backend (llama.cpp, LM Studio, Ollama, text-generation-webui). This model uses Qwen's thinking format -- it will produce reasoning tokens before its response. Give it sufficient generation budget (the reasoning pass typically uses hundreds to thousands of tokens before answering).

```bash
llama-cli -m Qwen3.6-14B-A3B-FableVibes-Q4_K_M.gguf \
  --mmproj Qwen3.6-14B-A3B-FableVibes-mmproj-F16.gguf \
  -p "Your prompt here"
```

## Notes

- This is a general-purpose reasoning model. The Fable traces improve structured thinking across domains.
- The pruned base was not pre-fine-tuned before this run -- the Fable LoRA was applied directly to the REAP output.
- Expect longer first-token latency due to the thinking pass, but higher quality reasoning on complex tasks.