Wondernutts/Qwen3.6-35B-A3B-StyleTune-int4-ov

Qwen3.6 35B-A3B StyleTune — OpenVINO INT4

OpenVINO INT4 conversion of Gryphe/Qwen3.6-35B-A3B-StyleTune for Intel CPU/GPU inference.

Gryphe created and trained the StyleTune. Wondernutts performed only the OpenVINO conversion. See the original card for the source methodology, benchmark details, notes, and credits.

About the original tune

This is Gryphe's Qwen version of a surgical style tune: only the lm_head output projection was trained while the transformer body remained frozen. Gryphe reports 46.9% fewer clichés and substantially different phrase selection on a 200-prompt roleplay benchmark, with the underlying Qwen capabilities retained. Those figures are from the source model and have not been rerun on this INT4 conversion.

The intended strengths are roleplay, character voice, narrative prose, and creative writing. Qwen3.6's native chat template applies automatically, and thinking mode is enabled by default in the source template.

Conversion details

Item	Value
Source	Gryphe/Qwen3.6-35B-A3B-StyleTune
Architecture	Qwen3.6 35B-A3B MoE conditional-generation/VLM export
Language-model compression	INT4 asymmetric, group size 64, ratio 1.0
Backup precision	INT8 symmetric
AWQ / ignored scope	none
OpenVINO binary payload	approximately 18.27 GiB
Export stack	Optimum 2.2 development exporter, Transformers 5.2, OpenVINO IR

The repository includes language, text-embedding, vision, tokenizer, and detokenizer IRs. The source configuration advertises a very large architectural context window; this specific conversion has not been independently validated across the full window.

Download

Download the public repository:

python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/Qwen3.6-35B-A3B-StyleTune-int4-ov --local-dir ./qwen36-35b-a3b-styletune-int4-ov

Run on Intel GPU

import openvino_genai as ov_genai

model_dir = "./qwen36-35b-a3b-styletune-int4-ov"
pipe = ov_genai.VLMPipeline(model_dir, "GPU", CACHE_DIR="./ov_cache")

config = ov_genai.GenerationConfig()
config.max_new_tokens = 1024
config.do_sample = True
config.temperature = 1.0
config.top_p = 0.95
config.repetition_penalty = 1.1

pipe.start_chat("You are a dry-witted companion in a character-driven fantasy story.")
print(pipe.generate("We have been walking this frozen pass for six hours. Say something.", generation_config=config))
pipe.finish_chat()

Use VLMPipeline, including for text-only chat, because the exported model is a conditional-generation/VLM graph. start_chat() applies the bundled Qwen template and retains KV cache. Gryphe reports using temperature 1.0, Min-P 0.10, and DRY on the source model; Min-P and DRY support depends on the serving layer.

Thinking is enabled by the source template by default. Allow enough output tokens for a complete reasoning-plus-answer response. For continuous batching on Intel GPU, set DYNAMIC_QUANTIZATION_GROUP_SIZE=0 if the dynamic activation-quantization path causes garbled output. Vision and performance have not been independently benchmarked for this exact conversion.

License and attribution

Apache-2.0, following Gryphe's source repository and upstream Qwen. StyleTune authorship and training credit: Gryphe. OpenVINO conversion: Wondernutts.