README.md

---
base_model: prism-ml/Bonsai-27B-gguf
license: apache-2.0
pipeline_tag: text-generation
language:
- en
tags:
- bonsai
- qwen3_5
- antidoom
- ftpo
- anti-repetition
- 1-bit
- q1_0
- gguf
- speculative-decoding
- dspark
- llama.cpp
- blackwell
---

# Bonsai-27B-antidoom-1bit-DSpark

A **[DSpark](https://github.com/PrismML-Eng/Bonsai-demo)-accelerated + [antidoom](https://github.com/Liquid4All/antidoom)
(FTPO anti-repetition)** build of [`prism-ml/Bonsai-27B`](https://huggingface.co/prism-ml/Bonsai-27B-gguf) —
the 1-bit (`Q1_0`, 1.125 bpw g128) quantization of the Qwen3.5-27B hybrid-attention model.

Runs in the **[PrismML-Eng/llama.cpp](https://github.com/PrismML-Eng/llama.cpp)** fork (CUDA, the
Q1_0 hybrid-attention kernels). Benchmarked on a single **RTX 5090 (32 GB, Blackwell sm_120)**.

## Contents
| File | What |
|---|---|
| `Bonsai-27B-antidoom-1bit-Q1_0.gguf` | antidoom-tuned 1-bit model (4.5 GB) |
| `Bonsai-27B-dspark-Q4_1.gguf` | DSpark speculative drafter (from prism-ml) |
| `Bonsai-27B-mmproj-Q8_0.gguf` | vision projector (from prism-ml) |

## Run with DSpark (fastest)
```bash
build/bin/llama-server \
  -m Bonsai-27B-antidoom-1bit-Q1_0.gguf \
  -md Bonsai-27B-dspark-Q4_1.gguf \
  --spec-type draft-dspark --spec-draft-n-max 4 \
  -ngl 999 -ngld 999 -fa on -c 16384 -np 1
```

## Speed comparison — all four models (RTX 5090, 32 GB)

Single stream, greedy, 256 new tokens, PrismML llama.cpp fork (CUDA 13 / sm_120).
Native = `llama-bench` tg256 (fa=1, r=2). DSpark = `llama-speculative-simple`,
`--spec-draft-n-max 4`, code prompt (high-acceptance workload); all four runs measured
back-to-back in the same session.

| Model | Native tok/s | + DSpark tok/s | Accept | DSpark speedup |
|---|---:|---:|---:|---:|
| Bonsai-27B **1-bit** (base) | 166.0 | **208.3** | 78.6% | 1.25× |
| **Bonsai-27B 1-bit antidoom** (this family) | 153.7 | **192.4** | 74.6% | 1.25× |
| Ternary-Bonsai-27B (base) | 136.5 | **185.9** | 69.2% | 1.36× |
| **Ternary-Bonsai-27B antidoom** (this family) | 129.2 | **184.6** | 69.2% | 1.43× |

```
tok/s (code prompt, bs=1)                 native ▒   +DSpark █
Bonsai 1-bit base      ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 166.0
                       ████████████████████████████████ 208.3
Bonsai 1-bit antidoom  ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 153.7
                       █████████████████████████████ 192.4
Ternary base           ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 136.5
                       ████████████████████████████ 185.9
Ternary antidoom       ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ 129.2
                       ████████████████████████████ 184.6
```

Notes:
- DSpark's gain is workload-dependent: on low-acceptance prose (~51%) the 1-bit target can be
  *slower* than native because the tiny weights make drafting overhead relatively expensive.
- The antidoom variants are ~5–8% slower natively because re-quantization stored the output
  tensor at q6_K (prism's originals pack it smaller); with DSpark the gap mostly closes.
- FTPO does not hurt the DSpark drafter (accept 74.6–78.6% 1-bit, 69.2% ternary, unchanged
  base vs antidoom for ternary).
- All numbers beat prism-ml's published H100 figures (104.8→143.8 1-bit, 98.0→131.8 ternary):
  these models are memory-bandwidth-bound and the 5090 has more bandwidth headroom at bs=1.

## Antidoom (FTPO) pass — what was done
Bonsai-27B is unusually **doom-loop resistant** (it self-corrects repetition and stays coherent even
under forced long generation), so runaway-repetition pairs were surfaced by **priming**: contexts that
have already begun repeating a phrase, sent to the raw completion endpoint, where the model continues
the loop. Antidoom's own detector + chosen-token sampler then extracted **40 FTPO preference pairs**
(rejected = the loop-continuation token, chosen = coherent escapes). FTPO trained a LoRA
(r=32, q/k/v/o + gate/up/down, 12 epochs, lr 3e-5) on the FP16 unpacked model:

- **chosen-win (prefers the coherent escape over continuing the loop): 0.11 → 0.66** (early-stopped).
- LoRA merged (CPU) into FP16, re-quantized to `Q1_0`. Note the aggressive 1-bit format partially
  dilutes the LoRA's fine adjustments, so the anti-repetition effect is strongest at higher precision.

Output remains coherent at 1-bit. This is a demonstration of the antidoom method on a
1-bit hybrid-attention target, packaged with the shipped DSpark drafter.

## Provenance
Base weights, DSpark drafter and mmproj are from [prism-ml](https://huggingface.co/prism-ml)
(re-quantized after the antidoom LoRA merge for the model weights; drafter/mmproj copied unchanged).
Built with [antidoom](https://github.com/Liquid4All/antidoom) and the PrismML llama.cpp fork.