Hugging Face's logo

FINAL-Bench
/
POCKET-26B-GGUF

Text Generation
GGUF
llama.cpp
conversational
on-device
mobile
korean
korean-llm
cpu
local-llm
edge
gemma
gemma4
mixture-of-experts
Mixture of Experts
pocket
vidraft
imatrix
Model card Files
xet
 Community
POCKET-26B-GGUF
File size: 6,503 Bytes
550b2ec
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9e5ad28
550b2ec
 
9e5ad28
550b2ec
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
---
license: apache-2.0
library_name: llama.cpp
pipeline_tag: text-generation
base_model:
- google/gemma-4-26B-A4B-it
tags:
- gguf
- llama.cpp
- conversational
- on-device
- mobile
- korean
- korean-llm
- cpu
- local-llm
- edge
- gemma
- gemma4
- mixture-of-experts
- moe
- pocket
- vidraft
---

> ### πŸ“š Collections
> **β–Ά [POCKET Models](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)** β€” this family (on-device, no GPU)
> [Darwin Family](https://huggingface.co/collections/FINAL-Bench/darwin-family-699987b1f652864af0122193) Β· [Aether Foundation](https://huggingface.co/collections/FINAL-Bench/aether-foundation-model-6a5c7f2fa1a4165c0414e53a) Β· [VKAE Accelerated](https://huggingface.co/collections/FINAL-Bench/vkae-accelerated-6a47231d7e7999dd8227675a) Β· [Metacognition Adapters](https://huggingface.co/collections/FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961)

![POCKET-26B](./pocket26_hero.svg)

# POCKET-26B-GGUF Β· ν•œκ΅­μ–΄

### A **Gemma4-26B-A4B**-based pocket model that loads in **any app today** β€” Ollama, LM Studio, PocketPal β€” with **no bleeding-edge runtime** needed. Korean-tuned, GPU-optional.

> πŸš€ **Try it live on a CPU (no GPU), no install β†’** [![POCKET-26B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--26B_CPU_chat-0f9d6e)](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) [![POCKET-35B demo](https://img.shields.io/badge/πŸ€—_Space-POCKET--35B_CPU_chat-ffce3a)](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU)

[![License](https://img.shields.io/badge/License-Apache_2.0-0f6e56)](https://www.apache.org/licenses/LICENSE-2.0) [![Runtime](https://img.shields.io/badge/runtime-any_llama.cpp-1baf7a)](https://github.com/ggml-org/llama.cpp) [![Compat](https://img.shields.io/badge/loads_in-Ollama_Β·_LM_Studio_Β·_PocketPal-2a9d8f)]() [![Base](https://img.shields.io/badge/base-Gemma--4--26B--A4B-185fa5)](https://huggingface.co/google/gemma-4-26B-A4B-it)

**Pick your build β†’** [![35B](https://img.shields.io/badge/POCKET--35B-GGUF-243456)](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) [![KR GGUF](https://img.shields.io/badge/POCKET--KR-GGUF-7A1F3D)](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) [![KR MLX](https://img.shields.io/badge/POCKET--KR-MLX-0f6e56)](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) [![EN GGUF](https://img.shields.io/badge/POCKET--EN-GGUF-185fa5)](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF)

## Why this one?

POCKET-26B takes Google's **Gemma4-26B-A4B** (25.2B total, ~4B active MoE, Apache-2.0) and re-quantizes it with our **proprietary Korean-tuned quantization** β€” **unpruned**, so quality holds. Unlike our Qwen-based POCKET (which needs a very recent `llama.cpp` build for its `qwen35moe` architecture), Gemma4 loads in **every mainstream runtime today**: Ollama, LM Studio, PocketPal, koboldcpp, and the browser.

## Quality β€” GPQA-Diamond, greedy, 198 questions (our harness)

| Build | GPQA-Diamond | vs base |
|---|---|---|
| Gemma4-26B-A4B (base) | 67.7% | β€” |
| **POCKET-26B `Q4_K_M`** | **67.7%** | **= base (lossless)** |
| **POCKET-26B `Q2_K`** (mixed) ⭐ | **67.2%** | βˆ’0.5pp (β‰ˆ lossless) |

*Single greedy pass, 198 items β†’ Β±~3 pp noise. Our proprietary Korean-tuned quantization is statistically lossless vs the base.*

## Files in this repo

| File | Size | Runs on | Best for |
|---|---|---|---|
| `POCKET-26B-Q4_K_M.gguf` | 17 GB | PC / high-RAM | top quality |
| **`POCKET-26B-Q2_K.gguf`** ⭐ | 11 GB | 12 GB phone / PC / browser | **universal daily driver** |

> Our **mixed-precision quantization** keeps the most quality-critical weights at higher precision β€” that is why `Q2_K` holds **67.2%** while a *plain* uniform Q2 collapses to ~44%.

## Quickstart β€” loads anywhere

```bash
# stock llama.cpp β€” brew / winget / apt, or LM Studio / Ollama / PocketPal
llama-cli -m POCKET-26B-Q2_K.gguf -p "λŒ€ν•œλ―Όκ΅­μ˜ μˆ˜λ„λŠ”?" -ngl 0 -t 8
```
No fork, no bleeding-edge build β€” Gemma4 support has shipped in every mainstream runtime since April 2026.

## Lineage (honest)

Based on **[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it)** (Apache-2.0). We do **not** re-host it unchanged β€” we add our proprietary **Korean-tuned quantization** (VIDRAFT). We deliberately **do not prune** it: Gemma4's low-bit robustness collapses under pruning (measured), so we keep all 128 experts and win on quality + universal compatibility instead.

## Limitations

- For **8 GB phones** (~5 GB budget), use [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) (5.1 GB) β€” Gemma4 cannot be shrunk that far without collapse.
- On-device iPhone/Mac throughput **not yet measured by us** β€” community reports welcome.

## Learn more

- Why on-device LLMs matter, and how POCKET measures up: [Can you run a large LLM without a GPU?](https://vidraft.net/insights/on-device-llm-without-gpu.html)
- What model quantization is, and why a 4-bit model stays smart: [What is model quantization?](https://vidraft.net/insights/what-is-quantization-llm.html)

## License
Apache-2.0 β€” use, modify, redistribute freely.

---
*POCKET is a VIDRAFT model family. Runs anywhere, no GPU.*

<!-- POCKET-FAMILY -->

---

## 🧩 The POCKET Family β€” On-device AI by VIDRAFT

*Big models, small hardware. No GPU, no cloud.*

**Models**
- πŸ“¦ [POCKET-35B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF) β€” flagship, PC / server, no GPU
- πŸ“¦ [POCKET-26B-GGUF](https://huggingface.co/FINAL-Bench/POCKET-26B-GGUF) β€” compact 26B
- πŸ‡°πŸ‡· [POCKET-KR-GGUF](https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF) β€” Korean, Android
- 🍎 [POCKET-KR-MLX](https://huggingface.co/FINAL-Bench/POCKET-KR-MLX) β€” Korean, iPhone / Mac
- 🌍 [POCKET-EN-GGUF](https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF) β€” English, phone / PC
- πŸ–ΌοΈ [POCKET-Image-Zimage](https://huggingface.co/FINAL-Bench/POCKET-Image-Zimage) β€” character-perfect text in any image

**Demos & tools (Spaces)**
- 🎨 [POCKET-Image Studio](https://huggingface.co/spaces/FINAL-Bench/POCKET-Image-Studio) β€” text-in-image, generate in-page
- πŸ–₯️ [POCKET-35B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU) β€” 35B answering on a CPU
- πŸ–₯️ [POCKET-26B-CPU](https://huggingface.co/spaces/FINAL-Bench/POCKET-26B-CPU) β€” 26B on a CPU

πŸ“š [Full POCKET collection](https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6)

<!-- /POCKET-FAMILY -->