Wondernutts/Twisted-Cyclone-31B-int4-ov

OpenVINO INT4 AWQ — asymmetric 4-bit weight compression, verified group size 128.

Twisted Cyclone 31B — OpenVINO INT4 AWQ

OpenVINO INT4 deployment conversion of Cyclone-Labs/Twisted-Cyclone-31B, prepared and validated by Wondernutts for Intel Arc and compatible OpenVINO devices.

This repository is a quantized deployment artifact. The original model design, merge, and fine-tune work belong to Cyclone-Labs and the source-model authors; Wondernutts performed the OpenVINO conversion and deployment validation and does not claim authorship of the source model work.

Source model and credits

Cyclone-Labs created Twisted Cyclone 31B from:

• google/gemma-4-31B-it
• Ateron/Gemma-4-Novelist-Eclipse-31B
• Vortex5/Glimmering-Citrus-31B
• Gryphe/Pantheon-Reasoning-31B-1.1
• Nimbz/Versipellis-31B

See the source model card for the merge configuration, intended use, original presentation, and complete upstream context.

OpenVINO conversion

• Weight format: asymmetric INT4
• Compression: AWQ, group size 128, ratio 1.0
• Graph: multimodal dense Gemma 4 OpenVINO IR (VLMPipeline)
• Tokenizer, detokenizer, text-embedding, and vision-embedding OpenVINO artifacts included
• RoPE lookup-table optimization: 131,072 positions, clamped at index 131,071 (LUT131K)
• Validation: artifact/configuration checks and the conversion pipeline's coherent-generation inference gate passed before publication

The repository's openvino_config.json records INT4 AWQ, group size 128, dynamic-quantization group size 128, ratio 1.0, and asymmetric weights. The LUT table is the hard positional ceiling of this exported graph. A deployed context window may be lower depending on GPU memory, KV-cache precision, scheduler, and memory budget.

Dense-31B reference performance

Wondernutts' current dense-31B reference test on one Intel Arc Pro B70 and the validated stock OpenVINO 2026.2 path recorded:

Reference test	Result
Prompt processing, 512 tokens	1,662 tok/s
Short-context decode	about 27 tok/s
Decode with about 6K tokens of context	about 19 tok/s
16K prompt prefill	about 43 seconds

These are platform/reference figures from the current local test corpus, not a benchmark claim for this individual model and not a universal or world-record claim. Performance varies with hardware, drivers, runtime, prompt shape, cache state, and generation settings.

The dense-31B figures were measured on the stock OpenVINO 2026.2 compatibility path. They are not attributed to the custom 26B MoE PA/XMX fork without a controlled dense-31B A/B test.

Wondernutts OpenVINO work

• Gemma 4 conversion, LUT, validation, and serving tooling: Wondernuttz/OpenVino-For-Gemma-4
• Custom Gemma 4 26B-A4B PA/XMX runtime work: Wondernuttz/openvino — arc-xe2-gemma4-pa-2026.4

The second link documents the custom 26B MoE runtime work; it is provenance, not the source of the dense-31B reference figures above.

Inference with OpenVINO GenAI

The validated dense-31B reference path used OpenVINO GenAI 2026.2:

Server-tested RP sampler

Use thinking OFF with temperature=0.69, top_p=0.95, top_k=64, min_p=0.05, and repetition_penalty=1.1. The server caps repetition penalty at 1.1. top_k=64 comes from these converted models' generation configurations; the remaining values are the controlled Chimera roleplay baseline used by the server. DRY and XTC are deliberately omitted because this OpenVINO deployment does not claim support for them.

import openvino_genai as ov_genai
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Wondernutts/Twisted-Cyclone-31B-int4-ov")

pipe = ov_genai.VLMPipeline(
    model_dir,
    "GPU",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.69
config.top_p = 0.95
config.top_k = 64
config.min_p = 0.05
config.repetition_penalty = 1.1
config.apply_chat_template = False

prompt = (
    "<bos>"
    "<|turn>system\nYou are a vivid, consistent roleplay partner.<turn|>\n"
    "<|turn>user\nWrite a short scene in a candlelit inn.<turn|>\n"
    "<|turn>model\n"
    "<|channel>thought\n<channel|>"
)

result = pipe.generate(prompt, generation_config=config)
print(result)

The final pre-closed thought channel requests a direct answer without exposing internal reasoning. Applications should keep internal reasoning hidden from end users.

Notes

• This is an OpenVINO deployment repository, not a Transformers checkpoint. Use OpenVINO/OpenVINO GenAI rather than AutoModelForCausalLM.
• Vision embeddings are included. Native image input requires the multimodal VLMPipeline API and a compatible OpenVINO GenAI build.
• License: Apache-2.0, inherited from the source model. Review the source and component cards before redistribution or commercial deployment.