Add README with validation results
Browse files
README.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gemma
|
| 3 |
+
base_model: shleekr/gemma4-expAF-ckpt486-merged
|
| 4 |
+
tags:
|
| 5 |
+
- roleplay
|
| 6 |
+
- storytelling
|
| 7 |
+
- korean
|
| 8 |
+
- japanese
|
| 9 |
+
- thai
|
| 10 |
+
- chinese
|
| 11 |
+
- w8a16
|
| 12 |
+
- compressed-tensors
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# Weave Pro 1.1 (expAF-ckpt486) — W8A16
|
| 16 |
+
|
| 17 |
+
Final flagship RP model (Weave Pro 1.1). Off-policy CPO-SimPO on expD base, batch-size/LR tuned (r=32, lr=5e-7, effective batch 16, constant schedule).
|
| 18 |
+
|
| 19 |
+
**Quantization**: W8A16 (weight-only INT8, per-channel, llmcompressor / compressed-tensors). `lm_head` and vision tower kept in bf16. Serve directly with vLLM.
|
| 20 |
+
|
| 21 |
+
## Why W8A16 (not 4-bit)
|
| 22 |
+
|
| 23 |
+
4-bit (AWQ W4A16, even with in-domain multilingual RP calibration) catastrophically degraded Korean morphology — corrupted particles/endings ("누구보다도도", "정말으로"), broken characters, and repetition loops. W8A16 fully preserves quality.
|
| 24 |
+
|
| 25 |
+
## Validation — win rate vs Gemini 2.5 Pro
|
| 26 |
+
|
| 27 |
+
Pairwise holistic judging (Claude Opus 4.6, temp 0, A/B randomized) on 200 real-service Korean RP prompts, identical prompts/judge across all rows:
|
| 28 |
+
|
| 29 |
+
| Variant | Win rate vs Gemini 2.5 Pro |
|
| 30 |
+
|---|---:|
|
| 31 |
+
| **This model (W8A16)** | **43.0%** |
|
| 32 |
+
| bf16 original | 47.0% (re-judged control: 46.2%) |
|
| 33 |
+
| 4-bit AWQ (rejected) | 18.6% |
|
| 34 |
+
|
| 35 |
+
Additional checks: defect rate on par with bf16 (broken chars: 0), Korean particle-corruption scan at bf16 level, side-by-side human reading indistinguishable from bf16.
|
| 36 |
+
|
| 37 |
+
## Usage (vLLM)
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
vllm serve shleekr/gemma4-expAF-ckpt486-w8a16 --max-model-len 49152 --dtype bfloat16
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
Includes the corrected `chat_template.jinja` (16,934 chars — with multi-turn continuation handling; older exports shipped a stale 12,045-char template).
|
| 44 |
+
|
| 45 |
+
Sampling used in validation: `temperature=0.8, top_p=0.9`.
|