Duplicate from Aatricks/bonsai-image-ternary-4B-FLUX2-klein-GGUF

Co-authored-by: Emilio Melis <Aatricks@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+bonsai-flux2-klein-ternary-q2_k.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,51 @@
+---
+license: apache-2.0
+base_model: prism-ml/bonsai-image-ternary-4B-unpacked
+tags:
+- flux
+- flux2
+- text-to-image
+- gguf
+- stable-diffusion.cpp
+- on-device
+- android
+library_name: gguf
+pipeline_tag: text-to-image
+---
+
+# Bonsai Image (Ternary, FLUX.2 Klein 4B) — Q2_K GGUF
+
+A GGUF conversion of **PrismML's Bonsai Image** ternary diffusion transformer
+([`prism-ml/bonsai-image-ternary-4B-unpacked`](https://huggingface.co/prism-ml/bonsai-image-ternary-4B-unpacked)),
+quantized to **Q2_K** so it runs on-device with
+[stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp) (and the
+[llmedge](https://github.com/Aatricks/llmedge) Android library).
+
+Bonsai Image is FLUX.2 Klein 4B fine-tuned with quantization-aware training (QAT) to ternary
+weights. Its official releases ship only in MLX (Apple) and GemLite (CUDA) packings, neither of
+which loads on Android. This is the **diffusion transformer only**, in the BFL tensor layout
+sdcpp expects, quantized to Q2_K (~1.3 GB).
+
+## Why Q2_K (not 1-bit)
+ggml's literal ternary types (`tq1_0`/`tq2_0`) load and run, but their per-256-weight scale is too
+coarse for Bonsai's per-128 trained scales and produce degraded output. **Q2_K**'s finer per-16
+sub-block scales preserve the QAT quality at a comparable size.
+
+## Usage (stable-diffusion.cpp)
+Also needs a Qwen3-4B text encoder GGUF and the FLUX.2 VAE:
+```bash
+sd --diffusion-model bonsai-flux2-klein-ternary-q2_k.gguf \
+   --vae flux2-vae.safetensors \
+   --llm qwen_3_4b.gguf \
+   -p "a red fox in snow, detailed, 8k" \
+   --cfg-scale 1.0 --steps 4 --offload-to-cpu --diffusion-fa --vae-tiling
+```
+
+## How it was made
+1. `scripts/convert_bonsai_flux2_to_bfl.py` (in llmedge): Bonsai diffusers transformer → BFL naming
+   (renames + fuses the double-block `to_q/k/v` into `*_attn.qkv`).
+2. `sd -M convert -m bonsai-flux2-bfl.safetensors --type q2_K -o <this file>`.
+
+## License
+Apache-2.0, inherited from the base model (PrismML / Deepgrove). All credit for the model and its
+QAT training goes to PrismML; this repo only redistributes a GGUF quantization.

bonsai-flux2-klein-ternary-q2_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:409a9db80c65d2c5b24f62a2fe0f4b3afc0e47965de9a29a7fe4614713398500
+size 1361094080