FaustusFaber31/gemma-4-E4B-it-awq-int4-xlam

🤖 Gemma AWQ (INT4) - Optimized for Tool Calling

Gemma-4-E4B-it-awq-int4-optimized is a highly specialized, quantized version of the Gemma instruction-tuned model. It has been compressed to 4-bit precision using AWQ (Activation-aware Weight Quantization) to maximize inference throughput and minimize VRAM usage, without sacrificing instruction-following capabilities.

---

🚀 Quickstart with vLLM

This model is fully optimized for high-performance serving with vLLM.

Installation

pip install vllm

Serving the Model

You can immediately deploy this model as an OpenAI-compatible API server:

python -m vllm.entrypoints.openai.api_server \
    --model Faustus-Faber/gemma-4-E4B-it-awq-int4-optimized \
    --quantization awq \
    --dtype auto \
    --max-model-len 4096

---

🧠 Model Details

• Base Model: Gemma Instruction-Tuned
• Quantization Method: AWQ (Activation-aware Weight Quantization)
• Precision: INT4 (Group Size: 128)
• Primary Use Case: Agentic workflows, tool-calling, and highly structured JSON generation.

🛠️ Intended Use

This model is designed to act as the reasoning engine for AI agents. The quantization profile exhibits high fidelity in generating structured JSON tool calls and following strict system prompts, even at extreme 4-bit compression.

---

📜 License & Citation

This model is subject to the standard Gemma Terms of Use.