Wondernutts/Lumimaid-Magnum-v4-12B-int4-ov

Lumimaid-Magnum v4 12B — OpenVINO INT4

OpenVINO INT4 conversion of Undi95/Lumimaid-Magnum-v4-12B, packaged for Intel CPU/GPU inference.

Undi95 created the source merge; Wondernutts performed only the OpenVINO conversion. Read the original model card and its linked component models for the authoritative lineage.

About the source model

The original is a DELLA/mergekit blend of Mistral Nemo 12B models, combining Lumimaid, Magnum v4, and Undi95's LocalC tune. The source card describes it as a requested Lumimaid/Magnum merge and notes that a Nemo tune trained on Claude input at 16K context was included in the mix.

It is aimed at roleplay, character dialogue, and creative writing. The prompt format is Mistral Instruct:

<s>[INST] {input} [/INST] {output}</s>

The shipped tokenizer contains the chat template, so normal OpenVINO GenAI chat mode applies it automatically.

Conversion details

Item	Value
Source	Undi95/Lumimaid-Magnum-v4-12B
Architecture	Text-only Mistral Nemo 12B
Weight compression	INT4 asymmetric, group size 128, ratio 1.0
OpenVINO binary payload	approximately 6.54 GiB
Included IRs	model, tokenizer, and detokenizer

The exported config advertises a 131,072-token architectural limit. That is not a claim of tested quality at 131K; the source card specifically mentions a 16K-trained merge component, and this conversion has not been independently validated across the full advertised window.

Download

Download the public repository:

python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/Lumimaid-Magnum-v4-12B-int4-ov --local-dir ./lumimaid-magnum-v4-12b-int4-ov

Run on Intel GPU

import openvino_genai as ov_genai

model_dir = "./lumimaid-magnum-v4-12b-int4-ov"
pipe = ov_genai.LLMPipeline(
    model_dir,
    "GPU",
    CACHE_DIR="./ov_cache",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.1

pipe.start_chat("You are a vivid, in-character roleplay partner.")
print(pipe.generate("The innkeeper slides a sealed letter across the bar.", generation_config=config))
pipe.finish_chat()

Use LLMPipeline, not VLMPipeline; this is a text-only export. start_chat() applies the bundled template and retains KV cache between turns. Change "GPU" to "CPU" for CPU inference.

Arc Pro B70 benchmark

Tested on one Intel Arc Pro B70 with prefix caching disabled. Each PP result is the mean of two exact-token runs after one same-shape warmup. Decode used 128 forced output tokens.

DQGS	512 PP tok/s	2K PP tok/s	6K PP tok/s	Decode tok/s
0	5,131	4,488	3,907	71.89
128	5,912	6,267	5,146	71.99

DQGS=128 improved PP by 15.2% at 512, 39.7% at 2K, and 31.7% at 6K. The generated-token hashes matched DQGS=0 at every measured shape, including the 128-token decode run.

License and attribution

The source repository does not declare a standalone license in its model-card metadata. Review the source and the terms of its component models before redistribution or commercial use. Source merge credit: Undi95. OpenVINO conversion: Wondernutts.