---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE
base_model: Qwen/Qwen2.5-VL-3B-Instruct
language:
- en
library_name: llama.cpp
pipeline_tag: image-text-to-text
tags:
- gguf
- qwen2_5_vl
- llama.cpp
- unsloth
- vision-language-model
- conversational
---

# KiwiMate Mini 1.0

KiwiMate Mini is the lightweight, on-device model tier for [KiwiMate](https://kiwimate.net), an AI companion app for iOS and Android with a New Zealand cultural identity. Mini 1.0 is fine-tuned from **Qwen2.5-VL-3B-Instruct**, replacing the previous Llama 3.2 3B base to add **image input support** alongside text chat.

This model is intended for fast, low-resource inference — locally on-device or via lightweight self-hosted endpoints — while KiwiMate Small (Qwen3.5-9B) handles heavier multimodal tasks server-side.

> ⚠️ **Preview build.** This is an early release of the Mini 1.0 line for internal testing ahead of KiwiMate app integration. Expect rough edges, especially around vision grounding and long conversations.

## What's new vs. Mini Preview (Llama 3.2 3B)

- **Vision input** — Mini can now see and reason about images (screenshots, photos, documents), not just text.
- New base architecture: `qwen2vl` (Qwen2.5-VL) instead of `llama`.
- Chat template moved from Llama-3's `<|eot_id|>` style to Qwen's ChatML-style `<|im_start|>` / `<|im_end|>` tokens — **update any client or server-side stop-sequence config accordingly** if you're migrating from Mini Preview.

## Files

| File | Size | Description |
|---|---|---|
| `KiwiMate-Mini-1.0-instruct.F16.gguf` | 6.18 GB | Full-precision weights, highest quality, largest footprint |
| `KiwiMate-Mini-1.0-instruct.Q6_K.gguf` | 2.54 GB | Near-lossless quantization, good balance of quality/size |
| `KiwiMate-Mini-1.0-instruct.Q4_K_M.gguf` | 1.93 GB | Smallest, fastest, recommended for constrained/on-device use |
| `KiwiMate-Mini-1.0.BF16-mmproj.gguf` | 1.34 GB | Vision projector — **required alongside a text GGUF above for image input** |
| `Modelfile` | — | Ollama Modelfile for local serving |
| `config.json` | — | Model config (Unsloth training output) |

As with the previous Mini line, expect some quality drop at Q4_K_M relative to F16/Q6_K — worth spot-checking once you've run your usual eval pass, since this is a different base architecture and the tradeoff may not track identically to the Llama 3.2 quants.

## Usage

### llama.cpp

Vision input requires loading **both** the text model and the mmproj file:

```bash
llama-mtmd-cli \
  -m KiwiMate-Mini-1.0-instruct.Q4_K_M.gguf \
  --mmproj KiwiMate-Mini-1.0.BF16-mmproj.gguf \
  -p "Describe this image."
```

### Ollama

```bash
ollama create kiwimate-mini -f Modelfile
ollama run kiwimate-mini
```

### Chat template

Qwen ChatML format:

```
<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
```

If you're serving this behind a proxy (e.g. `runpod-chat/index.ts`), make sure stop sequences are updated to `<|im_end|>` rather than the old Llama `<|eot_id|>` — the two are not interchangeable.

## Training

- **Base model:** [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct)
- **Framework:** [Unsloth](https://github.com/unslothai/unsloth) (`FastVisionModel`)
- **Quantization:** GGUF via llama.cpp (F16, Q6_K, Q4_K_M)
- **Fine-tuning focus:** New Zealand English, Te Reo Māori awareness and cultural sensitivity, KiwiMate persona and identity, conversational NZ context

## Intended use

Conversational AI companion for the KiwiMate app — casual chat, light image understanding (e.g. describing a photo, reading a screenshot), and NZ-flavoured personality. Not intended for high-stakes reasoning, factual research, or professional/medical/legal advice.

## License

⚠️ **Please read before redistributing.** This model is derived from Qwen2.5-VL-3B-Instruct, which Qwen distributes under a **Qwen Research license**, not Apache 2.0 — this differs from KiwiMate's other model tiers. Review the [license terms](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE) — research licenses commonly restrict commercial deployment — before treating this build the same as the Apache-2.0 Mini Preview / Small tiers, or before making this repo public.

## Links

- KiwiMate app: [kiwimate.net](https://kiwimate.net)
- Dev logs: [YouTube — KyleCodeKiwi](https://www.youtube.com/@KyleCodeKiwi)