--- license: apache-2.0 library_name: llama.cpp pipeline_tag: text-generation base_model: - google/gemma-4-26B-A4B-it tags: - gguf - llama.cpp - conversational - on-device - mobile - korean - korean-llm - cpu - local-llm - edge - gemma - gemma4 - mixture-of-experts - moe - pocket - vidraft --- > ### πŸ“š Collections > **β–Ά [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)** β€” this family (on-device, no GPU) > [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) Β· [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) Β· [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a) ![POCKET-26B](./pocket26_hero.svg) # POCKET-26B-GGUF Β· ν•œκ΅­μ–΄ ### A **Gemma4-26B-A4B**-based pocket model that loads in **any app today** β€” Ollama, LM Studio, PocketPal β€” with **no bleeding-edge runtime** needed. Korean-tuned, GPU-optional. > πŸš€ **Try it live on a CPU (no GPU), no install β†’** [![POCKET-26B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--26B_CPU_chat-0f9d6e)](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) [![POCKET-35B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--35B_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) [![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-any_llama.cpp-1baf7a)](https://github.com/ggml-org/llama.cpp) [![Compat](https://img.shields.io/badge/loads_in-Ollama_Β·_LM_Studio_Β·_PocketPal-2a9d8f)]() [![Base](https://img.shields.io/badge/base-Gemma--4--26B--A4B-185fa5)](https://huggingface.co/google/gemma-4-26B-A4B-it) **Pick your build β†’** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) ## Why this one? POCKET-26B takes Google's **Gemma4-26B-A4B** (25.2B total, ~4B active MoE, Apache-2.0) and re-quantizes it with our **proprietary Korean-tuned quantization** β€” **unpruned**, so quality holds. Unlike our Qwen-based POCKET (which needs a very recent `llama.cpp` build for its `qwen35moe` architecture), Gemma4 loads in **every mainstream runtime today**: Ollama, LM Studio, PocketPal, koboldcpp, and the browser. ## Quality β€” GPQA-Diamond, greedy, 198 questions (our harness) | Build | GPQA-Diamond | vs base | |---|---|---| | Gemma4-26B-A4B (base) | 67.7% | β€” | | **POCKET-26B `Q4_K_M`** | **67.7%** | **= base (lossless)** | | **POCKET-26B `Q2_K`** (mixed) ⭐ | **67.2%** | βˆ’0.5pp (β‰ˆ lossless) | *Single greedy pass, 198 items β†’ Β±~3 pp noise. Our proprietary Korean-tuned quantization is statistically lossless vs the base.* ## Files in this repo | File | Size | Runs on | Best for | |---|---|---|---| | `POCKET-26B-Q4_K_M.gguf` | 17 GB | PC / high-RAM | top quality | | **`POCKET-26B-Q2_K.gguf`** ⭐ | 11 GB | 12 GB phone / PC / browser | **universal daily driver** | > Our **mixed-precision quantization** keeps the most quality-critical weights at higher precision β€” that is why `Q2_K` holds **67.2%** while a *plain* uniform Q2 collapses to ~44%. ## Quickstart β€” loads anywhere ```bash # stock llama.cpp β€” brew / winget / apt, or LM Studio / Ollama / PocketPal llama-cli -m POCKET-26B-Q2_K.gguf -p "λŒ€ν•œλ―Όκ΅­μ˜ μˆ˜λ„λŠ”?" -ngl 0 -t 8 ``` No fork, no bleeding-edge build β€” Gemma4 support has shipped in every mainstream runtime since April 2026. ## Lineage (honest) Based on **[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)** (Apache-2.0). We do **not** re-host it unchanged β€” we add our proprietary **Korean-tuned quantization** (VIDRAFT). We deliberately **do not prune** it: Gemma4's low-bit robustness collapses under pruning (measured), so we keep all 128 experts and win on quality + universal compatibility instead. ## Limitations - For **8 GB phones** (~5 GB budget), use [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) (5.1 GB) β€” Gemma4 cannot be shrunk that far without collapse. - On-device iPhone/Mac throughput **not yet measured by us** β€” community reports welcome. ## Learn more - Why on-device LLMs matter, and how POCKET measures up: [Can you run a large LLM without a GPU?](https://vidraft.net/insights/on-device-llm-without-gpu.html) - What model quantization is, and why a 4-bit model stays smart: [What is model quantization?](https://vidraft.net/insights/what-is-quantization-llm.html) ## License Apache-2.0 β€” use, modify, redistribute freely. --- *POCKET is a VIDRAFT model family. Runs anywhere, no GPU.* --- ## 🧩 The POCKET Family β€” On-device AI by VIDRAFT *Big models, small hardware. No GPU, no cloud.* **Models** - πŸ“¦ [POCKET-35B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) β€” flagship, PC / server, no GPU - πŸ“¦ [POCKET-26B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) β€” compact 26B - πŸ‡°πŸ‡· [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) β€” Korean, Android - 🍎 [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) β€” Korean, iPhone / Mac - 🌍 [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) β€” English, phone / PC - πŸ–ΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) β€” character-perfect text in any image **Demos & tools (Spaces)** - 🎨 [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio) β€” text-in-image, generate in-page - πŸ–₯️ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) β€” 35B answering on a CPU - πŸ–₯️ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β€” 26B on a CPU πŸ“š [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)