shleekr/gemma4-simpo-expD-w8a16

Weave baseline (expD) — W8A16

Service baseline (Weave). SimPO format-corrected Gemma4 31B; base model of all subsequent experiments.

Quantization: W8A16 (weight-only INT8, per-channel, llmcompressor / compressed-tensors). lm_head and vision tower kept in bf16. Serve directly with vLLM.

Why W8A16 (not 4-bit)

4-bit (AWQ W4A16, even with in-domain multilingual RP calibration) catastrophically degraded Korean morphology — corrupted particles/endings ("누구보다도도", "정말으로"), broken characters, and repetition loops. W8A16 fully preserves quality.

Validation — win rate vs Gemini 2.5 Pro

Pairwise holistic judging (Claude Opus 4.6, temp 0, A/B randomized) on 200 real-service Korean RP prompts, identical prompts/judge across all rows:

Variant	Win rate vs Gemini 2.5 Pro
This model (W8A16)	40.6%
bf16 original	36.0%
4-bit AWQ (rejected)	20.3%

Additional checks: defect rate on par with bf16 (broken chars: 0), Korean particle-corruption scan at bf16 level, side-by-side human reading indistinguishable from bf16.

Usage (vLLM)

vllm serve shleekr/gemma4-simpo-expD-w8a16 --max-model-len 49152 --dtype bfloat16

Includes the corrected chat_template.jinja (16,934 chars — with multi-turn continuation handling; older exports shipped a stale 12,045-char template).

Sampling used in validation: temperature=0.8, top_p=0.9.