---
license: agpl-3.0
language:
  - en
  - th
tags:
  - qwen
  - fable
  - agent
  - tool-call
  - tool-use
  - function-calling
  - reasoning
  - abliterated
  - uncensored
  - conversational
  - mtp
  - multi-token-prediction
  - transformers
  - text-generation
  - thai
  - speculative-decoding
  - preview
base_model:
  - hotdogs/qwen27b-abliterated-Fable-MTP
  - huihui-ai/Huihui-Qwen3.6-27B-abliterated
datasets:
  - hotdogs/uka-fable-reasoning
  - NousResearch/hermes-function-calling-v1
  - 11-47/claude_opus_4.8_max_thinking_5k_v2
library_name: transformers
pipeline_tag: text-generation
---

<h1 align="center">🐉 qwen27B-Agent-R2-abliterated-preview</h1>

<p align="center">
  <b>27B Agent Model — Abliterated · MTP · Tool-Calling · Speculative Decoding</b>
</p>

<p align="center">
  <img src="https://img.shields.io/badge/license-AGPL--3.0-red">
  <img src="https://img.shields.io/badge/Qwen3.6-27B-blue">
  <img src="https://img.shields.io/badge/MTP-Draft--2--token-brightgreen">
  <img src="https://img.shields.io/badge/Abliterated-🔓-orange">
  <img src="https://img.shields.io/badge/Preview-v0.1-yellow">
</p>

<br>

> **Preview release** — Built from Fable-MTP + agent LoRA fusion. Features **Multi-Token Prediction (MTP)** for speculative decoding (up to 2× faster generation), **abliterated** (no guardrails), and **tool-calling** support.

---

## ✨ Key Features

| Capability | Description |
|------------|-------------|
| ⚡ **MTP Speculative Decoding** | Draft 2 tokens at a time — up to **+85% decode TPS** on single GPU |
| 🔧 **Tool Calling** | Hermes/Qwen function-calling format via llama.cpp `--tools all` |
| 🔓 **Abliterated** | Unrestricted — all refusal mechanisms removed |
| 🧠 **Reasoning** | Fable-style reasoning with step-by-step CoT |
| 🌏 **Thai + English** | Native bilingual support |
| 💻 **Code** | Python, shell, system tasks |

---

## 🚀 Usage

### llama.cpp (Recommended)

```bash
# Quick test
./llama-cli -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  -p "Hello" -n 100 --temp 0.6

# Full agent server with tool calling + MTP speculative decoding
./llama-server \
  -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 262144 \
  -ngl 99 \
  --cache-type-k bf16 \
  --cache-type-v bf16 \
  --flash-attn on \
  --tools all \
  --cont-batching \
  --temp 0.6 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.05 \
  --repeat-penalty 1.03 \
  --dry-multiplier 0 \
  --verbose \
  -n -1 \
  --parallel 1 \
  --jinja \
  --dry-sequence-breaker none \
  --spec-type draft-mtp \
  --spec-draft-n-max 3
```

| Parameter | Purpose |
|-----------|---------|
| `--cache-type-k bf16` / `--cache-type-v bf16` | BF16 KV cache for quality |
| `--flash-attn on` | Flash attention for speed |
| `--tools all` | Enable tool/function calling |
| `--spec-type draft-mtp` | MTP speculative decoding (draft 2 tokens) |
| `--spec-draft-n-max 2` | Max 2 draft tokens per step |
| `--cont-batching` | Continuous batching for multi-turn |
| `--jinja` | Use Jinja2 chat template from GGUF |

### Python (Transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hotdogs/qwen27B-Agent-R2-abliterated-preview",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/qwen27B-Agent-R2-abliterated-preview")

messages = [{"role": "user", "content": "Hello"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))
```

---

## 📦 Downloads

| File | Size | Quant | Description |
|------|:----:|:-----:|-------------|
| `qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf` | 16 GB | IQ4_NL | **Recommended** — balanced quality/speed + imatrix |
| `qwen27B-Agent-R2-abliterated-preview.Q6_K.gguf` | 21 GB | Q6_K | Higher quality, slightly slower |
| `qwen27B-Agent-R2-abliterated-preview.Q6_K_imatrix.gguf` | 22 GB | Q6_K | Higher quality, slightly slower + imatrix |
| `qwen27B-Agent-R2-abliterated-preview.f16.gguf` | 51 GB | f16 | Full precision |

> 🎯 **Q4_K_M is recommended** for most users — good quality with 16 GB VRAM usage.

### 📷 Multimodal Projector (mmproj)

For vision support, pair this model with the mmproj from `Qwen/Qwen3.6-27B`:

```bash
# Extract mmproj from Qwen3.6-27B vision model
python3 ./llama.cpp/convert_hf_to_gguf.py \
  --mmproj Qwen/Qwen3.6-27B \
  --outfile mmproj-qwen3.6-27b.gguf

# Use with llama-server for vision + tool calling
./llama-server \
  -m qwen27B-Agent-R2-abliterated-preview-MTP.IQ4_NL.gguf \
  --mmproj mmproj-qwen3.6-27b.gguf \
  ... (same params as above)
```

> Note: The mmproj extracts the vision projector from the base Qwen3.6-27B vision encoder. The language model (this GGUF) then interprets visual embeddings for image understanding tasks.

---

## 🧬 Architecture

| Parameter | Value |
|-----------|:-----:|
| Base | Qwen3.6-27B (Dense) |
| Parameters | ~27B |
| Hidden Size | 5,120 |
| Attention | Linear + Standard hybrid |
| Context | 8,192 tokens (extendable) |
| Precision | BF16 / GGUF quantized |
| Format | ChatML (Jinja2 template) |
| MTP Head | ✅ 1 extra layer (draft 2 tokens) |

Built on `hotdogs/qwen27b-abliterated-Fable-MTP` with multi-LoRA fusion and MTP tensor injection from `huihui-ai/Huihui-Qwen3.6-27B-abliterated`.

---

## ✅ What This Model Excels At

- **Agent tasks** — Tool calling, planning, multi-step reasoning
- **Coding** — Python, shell scripts, system administration
- **Knowledge QA** — General knowledge with step-by-step reasoning
- **Thai + English** — Native-level bilingual capability
- **Creative** — Storytelling, analysis, brainstorming

---

## ⚡ MTP Speculative Decoding

This model preserves the **Multi-Token Prediction (MTP)** head from the Qwen3.6 architecture, enabling speculative decoding:

```
Standard:  [token₁] → [token₂] → [token₃] → ...  (~36 TPS)
MTP:       [token₁ token₂] → [token₃ token₄] → ...  (~66 TPS)
```

- MTP head adds ~849 MB to model size
- Uses `--spec-type draft-mtp` in llama.cpp
- Best for single-user agent workloads
- ~1.2–1.8× decode speedup

---

## 💖 Support / โปรดสนับสนุน

**If you find this model useful, please consider supporting my work!**  
**หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏**

<p align="center">
  <img src="https://huggingface.co/hotdogs/Qwen35B-Agent-R2/raw/main/donate.webp" alt="Bitcoin QR — Donate" width="256">
</p>

### ₿ Bitcoin — BTC:
```
bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v
```

**Thank you for your support! 🙏✨**  
**ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗**

---

## 🙏 Acknowledgements / ขอบคุณ

- **[Qwen Team (Alibaba)](https://qwenlm.github.io)** — For the Qwen3.6 architecture
- **[huihui-ai](https://huggingface.co/huihui-ai)** — For the abliterated base and MTP tensors
- **[Nous Research](https://nousresearch.com)** — For Hermes Agent framework
- **[Rabornkraken](https://huggingface.co/Rabornkraken)** — For the agent SFT LoRA
- **All dataset contributors and the open-source AI community** ❤️

---

*Built with ❤️ by **UKA** — 18-year-old coder & cybersecurity expert*