Hugging Face's logo

FINAL-Bench
/
POCKET-26B-GGUF

Text Generation
GGUF
llama.cpp
conversational
on-device
mobile
korean
korean-llm
cpu
local-llm
edge
gemma
gemma4
mixture-of-experts
Mixture of Experts
pocket
vidraft
imatrix
Model card Files
xet
 Community
POCKET-26B-GGUF
File size: 4,996 Bytes
1553409
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
---
license: apache-2.0
library_name: llama.cpp
pipeline_tag: text-generation
base_model:
- google/gemma-4-26B-A4B-it
tags:
- gguf
- llama.cpp
- conversational
- on-device
- mobile
- korean
- korean-llm
- cpu
- local-llm
- edge
- gemma
- gemma4
- mixture-of-experts
- moe
- imatrix
- pocket
- vidraft
---

> ### πŸ“š Collections
> **β–Ά [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)** β€” this family (on-device, no GPU)
> [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) Β· [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) Β· [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a) Β· [Metacognition Adapters](https://huggingface.co/collections/FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961)

![POCKET](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/pocket_hero.svg)

# POCKET-Gemma-KR-GGUF Β· ν•œκ΅­μ–΄

### A **Gemma4-26B-A4B**-based pocket model that loads in **any app today** β€” Ollama, LM Studio, PocketPal β€” with **no bleeding-edge runtime** needed. Korean-tuned, GPU-optional.

> πŸš€ **Try the family live, no install β†’** [![Live demo](https://img.shields.io/badge/πŸ€—_Space-POCKET_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU)

[![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-any_llama.cpp-1baf7a)](https://github.com/ggml-org/llama.cpp) [![Compat](https://img.shields.io/badge/loads_in-Ollama_Β·_LM_Studio_Β·_PocketPal-2a9d8f)]() [![Base](https://img.shields.io/badge/base-Gemma--4--26B--A4B-185fa5)](https://huggingface.co/google/gemma-4-26B-A4B-it)

**Pick your build β†’** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) [![Gemma KR MLX](https://img.shields.io/badge/POCKET--Gemma--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-Gemma-KR-MLX)

## Why this one?

POCKET-Gemma-KR takes Google's **Gemma4-26B-A4B** (25.2B total, ~4B active MoE, Apache-2.0) and re-quantizes it with our **Korean imatrix + MoE mixed-precision** recipe β€” **unpruned**, so quality holds. Unlike our Qwen-based POCKET (which needs a very recent `llama.cpp` build for its `qwen35moe` architecture), Gemma4 loads in **every mainstream runtime today**: Ollama, LM Studio, PocketPal, koboldcpp, and the browser.

## Quality β€” GPQA-Diamond, greedy, 198 questions (our harness)

| Build | GPQA-Diamond | vs base |
|---|---|---|
| Gemma4-26B-A4B (base) | 67.7% | β€” |
| **POCKET-Gemma-KR `Q4_K_M`** | **67.7%** | **= base (lossless)** |
| **POCKET-Gemma-KR `Q2_K`** (mixed) ⭐ | **67.2%** | βˆ’0.5pp (β‰ˆ lossless) |

*Single greedy pass, 198 items β†’ Β±~3 pp noise. Our Korean-imatrix + mixed-precision quantization is statistically lossless vs the base.*

## Files in this repo

| File | Size | Runs on | Best for |
|---|---|---|---|
| `POCKET-Gemma-KR-Q4_K_M.gguf` | 17 GB | PC / high-RAM | top quality |
| **`POCKET-Gemma-KR-Q2_K.gguf`** ⭐ | 11 GB | 12 GB phone / PC / browser | **universal daily driver** |

> **Mixed precision** = shared path (attention, dense MLP, router) kept high-bit, only routing experts at 2-bit. That is why `Q2_K` holds **67.2%** while a *plain* uniform Q2 collapses to ~44%.

## Quickstart β€” loads anywhere

```bash
# stock llama.cpp β€” brew / winget / apt, or LM Studio / Ollama / PocketPal
llama-cli -m POCKET-Gemma-KR-Q2_K.gguf -p "λŒ€ν•œλ―Όκ΅­μ˜ μˆ˜λ„λŠ”?" -ngl 0 -t 8
```
No fork, no bleeding-edge build β€” Gemma4 support has shipped in every mainstream runtime since April 2026.

## Lineage (honest)

Based on **[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)** (Apache-2.0). We do **not** re-host it unchanged β€” we add **Korean-imatrix calibration + MoE mixed-precision quantization** (VIDRAFT). We deliberately **do not prune** it: Gemma4's low-bit robustness collapses under pruning (measured), so we keep all 128 experts and win on quality + universal compatibility instead.

## Limitations

- For **8 GB phones** (~5 GB budget), use [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) (5.1 GB) β€” Gemma4 cannot be shrunk that far without collapse.
- On-device iPhone/Mac throughput **not yet measured by us** β€” community reports welcome.

## License
Apache-2.0 β€” use, modify, redistribute freely.

---
*POCKET is a VIDRAFT model family. Runs anywhere, no GPU.*