---
license: apache-2.0
library_name: openvino
pipeline_tag: image-text-to-text
base_model:
- prism-ml/Ternary-Bonsai-27B-unpacked
tags:
- openvino
- int4
- intel-arc
- qwen3_5
- conversational
- bonsai
- ternary
---

# Ternary Bonsai 27B — OpenVINO INT4 SYM

This is an OpenVINO INT4 conversion of [PrismML's Ternary-Bonsai-27B unpacked checkpoint](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-unpacked), packaged for Intel CPU/GPU inference.

PrismML created and trained Ternary Bonsai. This repository contains only the OpenVINO conversion by Wondernutts. See the [original model card](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-unpacked) for the authoritative description and intended use.

## Important: this is not the native ternary build

The source checkpoint expands the ternary model into ordinary FP16 tensors for compatibility. This repository applies conventional OpenVINO INT4 compression to those unpacked tensors; it does **not** use PrismML's native ternary packing or custom kernels.

For the efficient native deployment, use PrismML's [ternary MLX](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-mlx-2bit) or [ternary GGUF](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf) release.

## Conversion details

| Item | Value |
|---|---|
| Source | `prism-ml/Ternary-Bonsai-27B-unpacked` |
| Architecture | Qwen3.5 conditional-generation/VLM graph |
| Weight compression | INT4 symmetric, group size 128 |
| Backup precision | INT8 symmetric |
| INT4 ratio | 1.0; all language-model layers requested |
| OpenVINO binary payload | approximately 13.94 GiB |
| Export stack | Optimum 2.2 development exporter, Transformers 5.2, OpenVINO IR |

The repository includes language, text-embedding, vision, tokenizer, and detokenizer IRs. The source configuration's maximum context has not been independently validated for this conversion.

## Download

Download the public repository:

```bash
python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/Ternary-Bonsai-27B-int4-sym-ov --local-dir ./ternary-bonsai-27b-int4-ov
```

## Run on Intel GPU

```python
import openvino_genai as ov_genai

model_dir = "./ternary-bonsai-27b-int4-ov"
pipe = ov_genai.VLMPipeline(model_dir, "GPU", CACHE_DIR="./ov_cache")

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.8
config.top_p = 0.95

pipe.start_chat("You are a thoughtful and precise assistant.")
print(pipe.generate("Write a short scene set in a city floating above the clouds.", generation_config=config))
pipe.finish_chat()
```

The shipped tokenizer applies the model's chat template. Change `"GPU"` to `"CPU"` for CPU inference. For continuous batching on Intel GPU, set `DYNAMIC_QUANTIZATION_GROUP_SIZE=0` if dynamic activation quantization causes garbled output.

For image input, load an RGB image into an `openvino.Tensor` and pass it as `images=[tensor]` to `VLMPipeline.generate()`. Vision and long-context behavior have not been independently benchmarked for this build.

## License and attribution

Apache-2.0, following the source repository. Model authorship and training credit belong to [PrismML](https://huggingface.co/prism-ml); OpenVINO conversion credit belongs to Wondernutts.