license: gemma
base_model: shleekr/gemma4-simpo-expD-merged
tags:
- roleplay
- storytelling
- korean
- japanese
- thai
- chinese
- w8a16
- compressed-tensors
Weave baseline (expD) — W8A16
Service baseline (Weave). SimPO format-corrected Gemma4 31B; base model of all subsequent experiments.
Quantization: W8A16 (weight-only INT8, per-channel, llmcompressor / compressed-tensors). lm_head and vision tower kept in bf16. Serve directly with vLLM.
Why W8A16 (not 4-bit)
4-bit (AWQ W4A16, even with in-domain multilingual RP calibration) catastrophically degraded Korean morphology — corrupted particles/endings ("누구보다도도", "정말으로"), broken characters, and repetition loops. W8A16 fully preserves quality.
Validation — win rate vs Gemini 2.5 Pro
Pairwise holistic judging (Claude Opus 4.6, temp 0, A/B randomized) on 200 real-service Korean RP prompts, identical prompts/judge across all rows:
| Variant | Win rate vs Gemini 2.5 Pro |
|---|---|
| This model (W8A16) | 40.6% |
| bf16 original | 36.0% |
| 4-bit AWQ (rejected) | 20.3% |
Additional checks: defect rate on par with bf16 (broken chars: 0), Korean particle-corruption scan at bf16 level, side-by-side human reading indistinguishable from bf16.
Usage (vLLM)
vllm serve shleekr/gemma4-simpo-expD-w8a16 --max-model-len 49152 --dtype bfloat16
Includes the corrected chat_template.jinja (16,934 chars — with multi-turn continuation handling; older exports shipped a stale 12,045-char template).
Sampling used in validation: temperature=0.8, top_p=0.9.