Aatricks/bonsai-image-ternary-4B-FLUX2-klein-GGUF

Bonsai Image (Ternary, FLUX.2 Klein 4B) — Q2_K GGUF

A GGUF conversion of PrismML's Bonsai Image ternary diffusion transformer (prism-ml/bonsai-image-ternary-4B-unpacked), quantized to Q2_K so it runs on-device with stable-diffusion.cpp (and the llmedge Android library).

Bonsai Image is FLUX.2 Klein 4B fine-tuned with quantization-aware training (QAT) to ternary weights. Its official releases ship only in MLX (Apple) and GemLite (CUDA) packings, neither of which loads on Android. This is the diffusion transformer only, in the BFL tensor layout sdcpp expects, quantized to Q2_K (~1.3 GB).

Why Q2_K (not 1-bit)

ggml's literal ternary types (tq1_0/tq2_0) load and run, but their per-256-weight scale is too coarse for Bonsai's per-128 trained scales and produce degraded output. Q2_K's finer per-16 sub-block scales preserve the QAT quality at a comparable size.

Usage (stable-diffusion.cpp)

Also needs a Qwen3-4B text encoder GGUF and the FLUX.2 VAE:

sd --diffusion-model bonsai-flux2-klein-ternary-q2_k.gguf \
   --vae flux2-vae.safetensors \
   --llm qwen_3_4b.gguf \
   -p "a red fox in snow, detailed, 8k" \
   --cfg-scale 1.0 --steps 4 --offload-to-cpu --diffusion-fa --vae-tiling

How it was made

1. scripts/convert_bonsai_flux2_to_bfl.py (in llmedge): Bonsai diffusers transformer → BFL naming (renames + fuses the double-block to_q/k/v into *_attn.qkv).
2. sd -M convert -m bonsai-flux2-bfl.safetensors --type q2_K -o <this file>.

License

Apache-2.0, inherited from the base model (PrismML / Deepgrove). All credit for the model and its QAT training goes to PrismML; this repo only redistributes a GGUF quantization.