---
license: apache-2.0
base_model:
  - Vortex5/Glimmering-Citrus-31B
base_model_relation: quantized
tags:
  - openvino
  - int4
  - awq
  - intel-arc
  - gemma4
  - roleplay
  - storytelling
  - conversational
pipeline_tag: image-text-to-text
---

> **OpenVINO INT4 AWQ** - asymmetric 4-bit weight compression, verified group size 128.


# Glimmering Citrus 31B — OpenVINO INT4 AWQ

OpenVINO INT4 deployment conversion of [Vortex5/Glimmering-Citrus-31B](https://huggingface.co/Vortex5/Glimmering-Citrus-31B), prepared and validated by Wondernutts for Intel Arc and compatible OpenVINO devices.

This repository is a quantized deployment artifact. The original model design, merge, and fine-tune work belong to Vortex5 and the source-model authors; Wondernutts performed the OpenVINO conversion and deployment validation and does not claim authorship of the source model work.

## Source model and credits

Vortex5 created Glimmering Citrus from:

- [ConicCat/Gemma4-Garnet-31B](https://huggingface.co/ConicCat/Gemma4-Garnet-31B)
- [zerofata/G4-MeroMero-v2-31B](https://huggingface.co/zerofata/G4-MeroMero-v2-31B)
- [LatitudeGames/Equinox-31B](https://huggingface.co/LatitudeGames/Equinox-31B)
- [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it)

See the [source model card](https://huggingface.co/Vortex5/Glimmering-Citrus-31B) for the merge configuration, intended use, original presentation, and complete upstream context.

## OpenVINO conversion

- Weight format: asymmetric INT4
- Compression: AWQ, group size 128, ratio 1.0
- Graph: multimodal Gemma 4 OpenVINO IR (`VLMPipeline`)
- Tokenizer and detokenizer OpenVINO artifacts included
- RoPE lookup-table optimization: 131,072 positions, clamped at index 131,071 (LUT131K)
- Validation: coherent mathematics and roleplay generation gates passed

The LUT table is the hard positional ceiling of this exported graph. A deployed context window may be lower depending on the GPU, KV-cache precision, scheduler, and memory budget.

## Dense-31B reference performance

The strongest dense-31B result documented in the current Wondernutts test corpus used one Intel Arc Pro B70 and the validated stock OpenVINO 2026.2 path:

| Reference test | Result |
|---|---:|
| Prompt processing, 512 tokens | **1,662 tok/s** |
| Short-context decode | **about 27 tok/s** |
| Decode with about 6K tokens of context | **about 19 tok/s** |
| 16K prompt prefill | **about 43 seconds** |

This is the highest result documented in the current local test corpus; it is **not** a universal, cross-hardware, or world-record claim. Performance varies with hardware, driver, runtime, prompt shape, cache state, and generation settings.

These dense-31B figures were measured on the stock OpenVINO 2026.2 compatibility path. They are not attributed to the custom 26B MoE PA/XMX fork unless a future controlled A/B test proves that fork is superior for dense 31B.

## Wondernutts OpenVINO work

- Gemma 4 conversion, LUT, validation, and serving tooling: [Wondernuttz/OpenVino-For-Gemma-4](https://github.com/Wondernuttz/OpenVino-For-Gemma-4)
- Custom Gemma 4 26B-A4B PA/XMX runtime work: [Wondernuttz/openvino — `arc-xe2-gemma4-pa-2026.4`](https://github.com/Wondernuttz/openvino/tree/arc-xe2-gemma4-pa-2026.4)

The second link documents the custom 26B MoE runtime work. It is provided for provenance and should not be read as the source of the dense-31B benchmark above.

## Inference with OpenVINO GenAI

The validated dense-31B reference path used OpenVINO GenAI 2026.2. Download the complete repository and point `VLMPipeline` at the local snapshot:

### Server-tested RP sampler

Use thinking **OFF** with `temperature=0.69`, `top_p=0.95`, `top_k=64`, `min_p=0.05`, and `repetition_penalty=1.1`. The server caps repetition penalty at 1.1. `top_k=64` comes from these converted models' generation configurations; the remaining values are the controlled Chimera roleplay baseline used by the server. DRY and XTC are deliberately omitted because this OpenVINO deployment does not claim support for them.

```python
import openvino_genai as ov_genai
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Wondernutts/Glimmering-Citrus-31B-int4-ov")

pipe = ov_genai.VLMPipeline(
    model_dir,
    "GPU",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.69
config.top_p = 0.95
config.top_k = 64
config.min_p = 0.05
config.repetition_penalty = 1.1
config.apply_chat_template = False

prompt = (
    "<bos>"
    "<|turn>system\nYou are a vivid, consistent roleplay partner.<turn|>\n"
    "<|turn>user\nWrite a short scene in a candlelit inn.<turn|>\n"
    "<|turn>model\n"
    "<|channel>thought\n<channel|>"
)

result = pipe.generate(prompt, generation_config=config)
print(result)
```

The final pre-closed thought channel requests a direct answer without exposing internal reasoning. Applications should keep any internal reasoning hidden from end users.

## Notes

- This is an OpenVINO deployment repository, not a Transformers checkpoint. Use OpenVINO/OpenVINO GenAI rather than `AutoModelForCausalLM`.
- The repository includes vision embeddings. Native image input requires the multimodal `VLMPipeline` API and a compatible OpenVINO GenAI build. Text inference is the path covered by the performance table.
- License: Apache-2.0, inherited from the source model. Review the source model and component cards before redistribution or commercial deployment.