---
license: gemma
base_model: shleekr/gemma4-multi-expQ-ckpt1584-merged
tags:
- roleplay
- storytelling
- korean
- japanese
- thai
- chinese
- w8a16
- compressed-tensors
---

# expQ-ckpt1584 — W8A16

Multilingual RP model. CPO-SimPO on expD base.

**Quantization**: W8A16 (weight-only INT8, per-channel, llmcompressor / compressed-tensors). `lm_head` and vision tower kept in bf16. Serve directly with vLLM.

## Why W8A16 (not 4-bit)

4-bit (AWQ W4A16, even with in-domain multilingual RP calibration) catastrophically degraded Korean morphology — corrupted particles/endings ("누구보다도도", "정말으로"), broken characters, and repetition loops. W8A16 fully preserves quality.

## Validation — win rate vs Gemini 2.5 Pro

Pairwise holistic judging (Claude Opus 4.6, temp 0, A/B randomized) on 200 real-service Korean RP prompts, identical prompts/judge across all rows:

| Variant | Win rate vs Gemini 2.5 Pro |
|---|---:|
| **This model (W8A16)** | **40.9%** |
| bf16 original | (not measured) |
| 4-bit AWQ (rejected) | 22.1% |

Additional checks: defect rate on par with bf16 (broken chars: 0), Korean particle-corruption scan at bf16 level, side-by-side human reading indistinguishable from bf16.

## Usage (vLLM)

```bash
vllm serve shleekr/gemma4-multi-expQ-ckpt1584-w8a16 --max-model-len 49152 --dtype bfloat16
```

Includes the corrected `chat_template.jinja` (16,934 chars — with multi-turn continuation handling; older exports shipped a stale 12,045-char template).

Sampling used in validation: `temperature=0.8, top_p=0.9`.