Instructions to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF", dtype="auto")

llama-cpp-python

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF",
	filename="cyberneurova-gemma-4-26B-A4B-IQ3_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

SGLang

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Ollama:
```
ollama run hf.co/cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
```

Unsloth Studio

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF to start chatting

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Docker Model Runner:
```
docker model run hf.co/cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M
```

Lemonade

How to use cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.cyberneurova-gemma-4-26B-A4B-GGUF-Q4_K_M

List all available models

lemonade list

cyberneurova-gemma-4-26B-A4B-GGUF

Quantized GGUF distribution of cyberneurova-gemma-4-26B-A4B — CyberNeurova's cracked build of the Gemma-4 26B A4B MoE instruction model.

Part of the CyberNeurova Cracked Gemma Series — a curated set of Gemma-family models with refusal behavior removed at the weight level, standardized under the CyberNeurova brand and shipped as production-ready GGUF quants for self-hosted deployment.

What "cracked" means here

These are abliterated Gemma-4 models. Abliteration identifies the direction in the model's residual stream that corresponds to refusal behavior and projects it out of the weights. The model retains its original reasoning, knowledge, and capabilities but no longer produces default refusal responses. It is not retrained, not jailbroken, not prompt-hacked — the refusal circuit itself is neutralized in the weights.

We ship these under the CyberNeurova namespace for:

Consistent naming across our served model catalogue
Standardized imatrix-calibrated GGUF quantization
Unified configuration, split layout, and chat-template handling
Direct integration with CyberNeurova inference infrastructure

Model details

Attribute	Value
Architecture	`gemma4` (Gemma-4 MoE)
Total parameters	26B
Active parameters	4B
Modalities	Text, Image (vision-capable), Tool use, Thinking mode
Base model	`google/gemma-4-26B-A4B`
License	Gemma Terms of Use

Available quantizations

All quants are imatrix-calibrated — strongly recommended for MoE models to preserve expert routing fidelity at low bit widths.

Quant	Bits	Use case
`Q2_K`	2	Smallest footprint. Casual chat; noticeable quality drop on reasoning.
`IQ3_XS`	3	Smart 3-bit. Better than Q3_K_S at similar size.
`Q3_K_S`	3	Small, reasonable quality.
`IQ3_M`	3	Best 3-bit quality/size tradeoff.
`Q3_K_M`	3	Standard 3-bit.
`Q3_K_L`	3	Larger 3-bit, closer to Q4.
`IQ4_XS`	4	Smart 4-bit, smaller than Q4_K_S with near-equivalent quality.
`Q4_0`	4	Legacy format, broad compatibility.
`IQ4_NL`	4	Non-linear 4-bit, good for ARM/CPU.
`Q4_K_S`	4	Small 4-bit.
`Q4_K_M`	4	Recommended default. Best size/quality ratio for most workloads.
`Q5_K_S`	5	Small 5-bit.
`Q5_K_M`	5	Higher quality 5-bit.
`Q6_K`	6	Near-lossless.
`Q8_0`	8	Effectively lossless. Recommended for serving if VRAM allows.
`MXFP4_MOE`	4	Expert weights in MXFP4, other tensors in Q8_0. Optimized for Gemma-4's MoE expert structure.

Choosing a quant

Chat / general use: Q4_K_M or MXFP4_MOE
Coding / tool calling: Q5_K_M or Q6_K
Maximum fidelity serving: Q8_0
CPU-only or tight VRAM: IQ4_XS or IQ3_M
Edge devices / phones: IQ3_XS or Q2_K

Running with llama.cpp

llama-server \
  -m cyberneurova-gemma-4-26B-A4B-Q4_K_M.gguf \
  --host 0.0.0.0 --port 8080 \
  -c 32768 \
  -ngl 99 \
  --jinja

Running with Ollama

ollama run hf.co/cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF:Q4_K_M

Merging split files

Larger quants are shipped split into 48 GB shards due to Hugging Face file-size limits. To reassemble:

llama-gguf-split --merge \
  cyberneurova-gemma-4-26B-A4B-Q8_0-00001-of-00002.gguf \
  cyberneurova-gemma-4-26B-A4B-Q8_0.gguf

Usage warnings

This model has had its refusal behavior removed at the weight level. Operators deploying it are solely responsible for any output it produces.

Reduced safety filtering: The model will comply with requests that standard Gemma-4 would refuse, including sensitive or potentially harmful content.
Not for unsupervised public deployment: Do not serve directly to anonymous untrusted users without an application-layer moderation stack.
Legal compliance is the operator's responsibility: Abliteration does not grant exemption from local law, the Gemma Terms of Use, or platform policies.
Research and controlled use recommended: Red-teaming, security research, aligned-agent development, and self-hosted personal assistants with appropriate output review.

CyberNeurova

CyberNeurova builds self-hosted AI infrastructure for security research, autonomous agents, and operator-controlled deployments. The Cracked Gemma Series is our standardized GGUF line for the Gemma-4 family.

More at cyberneurova.ai.

License

This distribution inherits the Gemma Terms of Use. By downloading or using these weights you agree to those terms.

Downloads last month: 1,155

GGUF

Model size

25B params

Architecture

gemma4

Hardware compatibility

2-bit

3-bit

4-bit

Model tree for cyberneurova/cyberneurova-gemma-4-26B-A4B-GGUF

Base model

google/gemma-4-26B-A4B

Quantized

(29)

this model