---
license: apache-2.0
library_name: llama.cpp
pipeline_tag: text-generation
base_model:
- google/gemma-4-26B-A4B-it
tags:
- gguf
- llama.cpp
- conversational
- on-device
- mobile
- korean
- korean-llm
- cpu
- local-llm
- edge
- gemma
- gemma4
- mixture-of-experts
- moe
- imatrix
- pocket
- vidraft
---

> ### 📚 Collections
> **▶ [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)** — this family (on-device, no GPU)
> [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) · [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) · [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a) · [Metacognition Adapters](https://huggingface.co/collections/FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961)

![POCKET](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/pocket_hero.svg)

# POCKET-Gemma-KR-GGUF · 한국어

### A **Gemma4-26B-A4B**-based pocket model that loads in **any app today** — Ollama, LM Studio, PocketPal — with **no bleeding-edge runtime** needed. Korean-tuned, GPU-optional.

> 🚀 **Try the family live, no install →** [![Live demo](https://img.shields.io/badge/🤗_Space-POCKET_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU)

[![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-any_llama.cpp-1baf7a)](https://github.com/ggml-org/llama.cpp) [![Compat](https://img.shields.io/badge/loads_in-Ollama_·_LM_Studio_·_PocketPal-2a9d8f)]() [![Base](https://img.shields.io/badge/base-Gemma--4--26B--A4B-185fa5)](https://huggingface.co/google/gemma-4-26B-A4B-it)

**Pick your build →** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) [![Gemma KR MLX](https://img.shields.io/badge/POCKET--Gemma--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-Gemma-KR-MLX)

## Why this one?

POCKET-Gemma-KR takes Google's **Gemma4-26B-A4B** (25.2B total, ~4B active MoE, Apache-2.0) and re-quantizes it with our **Korean imatrix + MoE mixed-precision** recipe — **unpruned**, so quality holds. Unlike our Qwen-based POCKET (which needs a very recent `llama.cpp` build for its `qwen35moe` architecture), Gemma4 loads in **every mainstream runtime today**: Ollama, LM Studio, PocketPal, koboldcpp, and the browser.

## Quality — GPQA-Diamond, greedy, 198 questions (our harness)

| Build | GPQA-Diamond | vs base |
|---|---|---|
| Gemma4-26B-A4B (base) | 67.7% | — |
| **POCKET-Gemma-KR `Q4_K_M`** | **67.7%** | **= base (lossless)** |
| **POCKET-Gemma-KR `Q2_K`** (mixed) ⭐ | **67.2%** | −0.5pp (≈ lossless) |

*Single greedy pass, 198 items → ±~3 pp noise. Our Korean-imatrix + mixed-precision quantization is statistically lossless vs the base.*

## Files in this repo

| File | Size | Runs on | Best for |
|---|---|---|---|
| `POCKET-Gemma-KR-Q4_K_M.gguf` | 17 GB | PC / high-RAM | top quality |
| **`POCKET-Gemma-KR-Q2_K.gguf`** ⭐ | 11 GB | 12 GB phone / PC / browser | **universal daily driver** |

> **Mixed precision** = shared path (attention, dense MLP, router) kept high-bit, only routing experts at 2-bit. That is why `Q2_K` holds **67.2%** while a *plain* uniform Q2 collapses to ~44%.

## Quickstart — loads anywhere

```bash
# stock llama.cpp — brew / winget / apt, or LM Studio / Ollama / PocketPal
llama-cli -m POCKET-Gemma-KR-Q2_K.gguf -p "대한민국의 수도는?" -ngl 0 -t 8
```
No fork, no bleeding-edge build — Gemma4 support has shipped in every mainstream runtime since April 2026.

## Lineage (honest)

Based on **[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)** (Apache-2.0). We do **not** re-host it unchanged — we add **Korean-imatrix calibration + MoE mixed-precision quantization** (VIDRAFT). We deliberately **do not prune** it: Gemma4's low-bit robustness collapses under pruning (measured), so we keep all 128 experts and win on quality + universal compatibility instead.

## Limitations

- For **8 GB phones** (~5 GB budget), use [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) (5.1 GB) — Gemma4 cannot be shrunk that far without collapse.
- On-device iPhone/Mac throughput **not yet measured by us** — community reports welcome.

## License
Apache-2.0 — use, modify, redistribute freely.

---
*POCKET is a VIDRAFT model family. Runs anywhere, no GPU.*