---
library_name: transformers
tags:
- gemma
- adelic
- topology
- infinite-context
- sparse-attention
- gguf
- llama-cpp
---

# Adelic-Gemma-4-31B-it

This repository contains the custom **Adèlic Cache** topological architecture wrapper for Gemma 4 (31B Multimodal), available in uncompressed and highly-optimized quantized (`Q4_K_M`, `Q5_K_M`) GGUF formats. 

By injecting the Adèlic `DynamicTopologyRouter` and Medoid-Value similarity clustering into the attention layers, this architecture aggressively condenses the Key-Value (KV) cache into a $p$-adic Bruhat-Tits tree. This bounds the physical VRAM footprint to $\mathcal{O}(\log N)$, allowing for **infinite context length generation on consumer hardware without Out-Of-Memory (OOM) crashes**.

You can run this model using two different methods:
1. **Natively via PyTorch/Transformers** using our custom SRAM-accelerated Triton kernel patch.
2. **Via `llama.cpp`** using the pre-compiled `.gguf` weights, which natively incorporates the topological condensation kernel directly into the CUDA backend.

---

## Method 1: PyTorch & Triton Patch (Native Hugging Face)

You do NOT need `trust_remote_code=True` because the patch applies cleanly onto native loaded models. Simply download the `patch_adelic.py` script from this repo and run it on your loaded model!

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import huggingface_hub

# 1. Download the Adèlic patch script
huggingface_hub.hf_hub_download(
    repo_id="sneedjak/Adelic-Gemma-4-31B-it", 
    filename="patch_adelic.py", 
    local_dir="."
)
from patch_adelic import apply_adelic_topology

# 2. Load the official Gemma tokenizer and model
model_id = "google/gemma-4-31B-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto"
)

# 3. Inject the Adèlic Topology (Triton Accelerated)
model = apply_adelic_topology(model)

# 4. Generate with infinite context!
prompt = "The quick brown fox jumps over the lazy dog. " * 50000 
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# The KV-cache will automatically condense, preventing your GPU from crashing.
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Method 2: llama.cpp (Pre-compiled GGUF)

For maximum efficiency and consumer hardware support, we have compiled the Adèlic condensation kernel natively into a custom fork of `llama.cpp` and uploaded the ready-to-run `.gguf` files to this repository.

### Available Quants
* `adelic-gemma4-31b.gguf` (Uncompressed `Q8`/`F16` Master Archive - Extremely High VRAM requirement)
* `adelic-gemma4-31b-Q5_K_M.gguf` (The "Sweet Spot" - Virtually indistinguishable from uncompressed, fits in 24GB VRAM)
* `adelic-gemma4-31b-Q4_K_M.gguf` (Fastest inference, lowest VRAM footprint)

> [!IMPORTANT]
> The `.gguf` files contained in this repository **require a custom fork** of `llama.cpp` to run, as standard `llama.cpp` does not have the `ggml_adelic_condense` CUDA kernel.

### Installation

1. Clone our custom `llama.cpp` fork containing the Adèlic kernels:
```bash
git clone -b feature/gemma4-adelic https://github.com/sneed-and-feed/llama.cpp.git
cd llama.cpp
```

2. Compile the binary (CUDA backend strictly required for the topological kernel!):
```bash
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j 8
```

3. Download your preferred `.gguf` quantization from this Hugging Face repository.

### Running Inference

Once compiled, you can run the model directly. The topology router will automatically and unconditionally mask out unneeded keys in the `kq_mask` on a per-layer basis using the hardware-accelerated CUDA condensation kernel!

```bash
./build/bin/llama-cli \
    -m adelic-gemma4-31b-Q4_K_M.gguf \
    -p "Explain the significance of the Bruhat-Tits tree in p-adic geometry." \
    -n 512 \
    -c 4096 \
    -ngl 999 \
    --color on
```

---

## Performance, Fixes & Limitations

* **Multimodal Hallucination Fix:** The custom `llama.cpp` engine associated with this repository includes a hardcoded `-INFINITY` logit bias suppression for Gemma 4's multimodal tokens (e.g. `<image|>`, `<audio|>`). This permanently cures the token leakage bug where Gemma models randomly hallucinate visual control tokens during long-context pure-text generation.
* **Semantic Fact Retrieval:** On the LongBench QASPER dataset, this architecture successfully retrieved grounded facts from 10,000+ tokens away despite the massive topological compression of the KV-cache, proving that the Medoid-Value clustering strategy preserves Strict RoPE Coherence.
* **Triton / CUDA Speedups:** The cache condensation runs completely $\mathcal{O}(1)$ inside SRAM, avoiding thousands of slow sequential Python loops, allowing for decoding speeds up to ~50 t/s even while actively pruning the cache.
* **Formatting Degradation:** Because topological compression is lossy, the model's surface-level syntactic formatting (e.g., RLHF alignment `<think>` tags) may degrade into a stream-of-consciousness format over extreme context lengths. While semantic facts are preserved, raw string-matching $n$-gram benchmark scores (like exact-match F1) may be lower than an uncompressed baseline. 

For full mathematical proofs of the RoPE coherence under topological compression, see the paper: *Llama Surgery: Injecting Differentiable p-Adic Topology into Pre-Trained LLMs*.