**ZwZ-2B-GGUF**

This repository provides GGUF-format weights for [ZwZ-2B](https://huggingface.co/inclusionAI/ZwZ-2B), split into two components:

 - Language model (LLM): FP16, Q8_0, Q4_K_M
 - Vision encoder (mmproj): FP16, Q8_0, Q4_K_M

These files are compatible with llama.cpp, Ollama, and other GGUF-based tools, supporting inference on CPU, NVIDIA GPU (CUDA), Apple Silicon (Metal), Intel GPUs (SYCL), and more. You can mix precision levels for the language and vision components based on your hardware and performance needs, and even perform custom quantization starting from the FP16 weights.

Enjoy running this multimodal model on your personal device! 🚀