---
library_name: transformers
tags:
- gemma
- adelic
- topology
- infinite-context
- sparse-attention
---

# Adelic-Gemma-4-31B-it

This repository contains the custom **Adèlic Cache** topological architecture wrapper for Gemma 4 (31B Multimodal). 

By injecting the Adèlic `DynamicTopologyRouter` and Medoid-Value similarity clustering into the attention layers, this architecture aggressively condenses the Key-Value (KV) cache into a $p$-adic Bruhat-Tits tree. This bounds the physical VRAM footprint to $\mathcal{O}(\log N)$, allowing for **infinite context length generation on consumer hardware without Out-Of-Memory (OOM) crashes**.

This repository is powered by a custom **Triton Kernel** that computes the memory condensation similarities directly inside the GPU SRAM, achieving FlashAttention-like speedups and completely avoiding intermediate memory allocations.

> [!NOTE]
> **Why does the model card say 0 parameters?**
> This repository only hosts the custom PyTorch patching script (`patch_adelic.py`). It does **not** re-host the massive 31GB Gemma 4 weights. You must load the official Google Gemma weights and inject this architecture at runtime (see usage below).

## Usage

You do NOT need `trust_remote_code=True` because the patch applies cleanly onto native loaded models. Simply download the `patch_adelic.py` script from this repo and run it on your loaded model!

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import huggingface_hub

# 1. Download the Adèlic patch script
huggingface_hub.hf_hub_download(
    repo_id="sneedjak/Adelic-Gemma-4-31B-it", 
    filename="patch_adelic.py", 
    local_dir="."
)
from patch_adelic import apply_adelic_topology

# 2. Load the official Gemma tokenizer and model
model_id = "google/gemma-4-31B-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    device_map="auto"
)

# 3. Inject the Adèlic Topology (Triton Accelerated)
model = apply_adelic_topology(model)

# 4. Generate with infinite context!
prompt = "The quick brown fox jumps over the lazy dog. " * 50000 
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# The KV-cache will automatically condense, preventing your GPU from crashing.
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Performance & Limitations

* **Semantic Fact Retrieval:** On the LongBench QASPER dataset, this architecture successfully retrieved grounded facts from 10,000+ tokens away despite the massive topological compression of the KV-cache.
* **Triton Speedup:** The cache condensation runs completely $\mathcal{O}(1)$ inside SRAM, avoiding thousands of slow sequential Python loops.
* **Formatting Degradation:** Because topological compression is lossy, the model's surface-level syntactic formatting (e.g., RLHF alignment `<think>` tags) degrades into a stream-of-consciousness format. While semantic facts are preserved, raw string-matching $n$-gram benchmark scores (like F1) will be lower than the uncompressed baseline. 

For full mathematical proofs of the RoPE coherence under topological compression, see the paper: *Llama Surgery: Injecting Differentiable p-Adic Topology into Pre-Trained LLMs*.