sneedjak/Adelic-Gemma-4-12B-GGUF

Adèlic Gemma 4 12B (GGUF)

This repository contains the GGUF quantized weights for google/gemma-4-12B, specifically optimized and patched to run on the experimental Adèlic Condense infinite-context engine.

🛠️ Changelog (V2 Patch)

• Native ChatML Integration: Addressed a bug where the model would occasionally hallucinate fake user dialogue (e.g., <|im_start|>user). This was traced to a mismatch between the base Gemma templates and the model's SFT format. The industry-standard ChatML Jinja2 template is now permanently baked into the GGUF metadata, allowing standard engines (and Ollama) to automatically parse the conversational stops natively.
• Native Multimodal Suppression: The <image|> and <audio|> generation hallucination bug has been permanently eradicated via the Adèlic engine and Hugging Face generation configurations.

🚀 The Adèlic Project

This model is designed to be run on a custom fork of llama.cpp that features the Adèlic KV Condenser.

The Adèlic engine utilizes a specialized CUDA kernel that continuously evaluates the cosine similarity of tokens inside the KV cache. By aggressively pruning older, redundant historical tokens (similarity > 0.999) from the value cache on every single layer, the engine theoretically enables an infinite context window without running out of VRAM or destroying attention coherence.

Engine-Level Patches Included

• Unconditional KV Condensation: The Adèlic condenser runs unconditionally across all layers to enforce unbounded context.
• Multimodal Hallucination Killswitch: The Adèlic inference engine hardcodes negative-infinity bias for image/audio modalities, permanently curing the checkpoint's hallucination issue at high Top K sampling.

💾 Available Quantizations

Filename	Format	Size	Recommended For
adelic-gemma4-12b-Q6_K.gguf	Q6_K	~9.8 GB	16GB VRAM GPUs (RTX 4080, T4, etc.)
adelic-gemma4-12b-patched.gguf	BF16/F16	~24 GB	24GB+ VRAM environments (A100, RTX 4090)

💻 How to Use

To use this model and activate the infinite-context condenser, you must use the official Adèlic fork of llama.cpp. Standard builds of llama.cpp will not have the required CUDA kernels for the infinite context window.

1. Clone & Build the Engine

# Clone the Adelic fork
git clone -b feature/gemma4-adelic https://github.com/sneed-and-feed/llama.cpp.git
cd llama.cpp

# Build with CUDA support (Includes both CLI and Server endpoints)
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j 4 --target llama-cli --target llama-server

2. Run Inference

Because the ChatML template is baked directly into the GGUF, you can run conversational mode natively without any clunky reverse-prompt flags!

Option A: Terminal CLI

./build/bin/llama-cli \
    -m /path/to/adelic-gemma4-12b-Q6_K.gguf \
    -n 512 \
    -c 4096 \
    -ngl 999 \
    -p "Explain the concept of quantum computing to a 5 year old:" \
    --color on

Option B: OpenAI-Compatible API Server If you want to use the infinite-context model with UIs like SillyTavern, Open WebUI, or AnythingLLM, you can spin up the local server endpoint:

./build/bin/llama-server \
    -m /path/to/adelic-gemma4-12b-Q6_K.gguf \
    -c 4096 \
    -ngl 999 \
    --port 8080

💡 Pro-Tip: Forcing the Reasoning Block If you notice the model is instantly closing its own reasoning block (outputting <|channel>thought followed immediately by <channel|>), you can physically force it to "think" by pre-filling the assistant's response.

Simply append <|channel>thought\n to the very end of your prompt! This drops the engine directly into the reasoning state, forcing it to generate logic tokens before answering.

📜 License

This model is subject to the official Gemma Terms of Use. By downloading or using these weights, you agree to the original Google Gemma License.