haven-ai-companion/gemma4-e4b-merged-iq4xs-turbo

Gemma 4 E4b Merged IQ4_XS (Turbo) GGUF

This is the custom-quantized Gemma 4 E4b Merged IQ4_XS (Turbo) model GGUF, specifically optimized for low-latency, high-performance interactions within the Haven AI Companion ecosystem.

Model Summary

• Base Architecture: Google Gemma 4 (E4b)
• Quantization Scheme: IQ4_XS (non-linear importance matrix quantization, preserving maximum weights coherence)
• Size: 5.09 GB
• Context Window: 16,384 tokens
• Use Case: Optimized for multi-character companion profiles, low-latency dialogue generation, and local text-to-speech alignment.

Setup Instructions

1. Download GGUF

To fetch the GGUF binary directly from the terminal:

wget https://huggingface.co/ssfdre38/gemma4-e4b-merged-iq4xs-turbo/resolve/main/gemma4-e4b-merged-iq4xs-turbo.gguf

2. Configure for Ollama

To compile a custom model within your local Ollama instance:

1. Create a file named Modelfile in the download directory:

   FROM ./gemma4-e4b-merged-iq4xs-turbo.gguf
   PARAMETER num_ctx 16384
   TEMPLATE """<start_of_turn>user
   {{ .Prompt }}<end_of_turn>
   <start_of_turn>model
   {{ .Response }}<end_of_turn>"""
   

2. Build the model:

   ollama create gemma4-turbo -f Modelfile
   

3. Verify it runs:

   ollama run gemma4-turbo
   

3. Connect to Haven Server

Update your appsettings.json inside your Haven Server config directory to point to the newly built model:

{
  "Ollama": {
    "BaseUrl": "http://localhost:11434",
    "Model": "gemma4-turbo"
  }
}

---

Built by the Haven AI Companion Project.