shleekr commited on
Commit
dbc8cd7
·
verified ·
1 Parent(s): ae2fdeb

Add README with validation results

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: gemma
3
+ base_model: shleekr/gemma4-expAF-ckpt486-merged
4
+ tags:
5
+ - roleplay
6
+ - storytelling
7
+ - korean
8
+ - japanese
9
+ - thai
10
+ - chinese
11
+ - w8a16
12
+ - compressed-tensors
13
+ ---
14
+
15
+ # Weave Pro 1.1 (expAF-ckpt486) — W8A16
16
+
17
+ Final flagship RP model (Weave Pro 1.1). Off-policy CPO-SimPO on expD base, batch-size/LR tuned (r=32, lr=5e-7, effective batch 16, constant schedule).
18
+
19
+ **Quantization**: W8A16 (weight-only INT8, per-channel, llmcompressor / compressed-tensors). `lm_head` and vision tower kept in bf16. Serve directly with vLLM.
20
+
21
+ ## Why W8A16 (not 4-bit)
22
+
23
+ 4-bit (AWQ W4A16, even with in-domain multilingual RP calibration) catastrophically degraded Korean morphology — corrupted particles/endings ("누구보다도도", "정말으로"), broken characters, and repetition loops. W8A16 fully preserves quality.
24
+
25
+ ## Validation — win rate vs Gemini 2.5 Pro
26
+
27
+ Pairwise holistic judging (Claude Opus 4.6, temp 0, A/B randomized) on 200 real-service Korean RP prompts, identical prompts/judge across all rows:
28
+
29
+ | Variant | Win rate vs Gemini 2.5 Pro |
30
+ |---|---:|
31
+ | **This model (W8A16)** | **43.0%** |
32
+ | bf16 original | 47.0% (re-judged control: 46.2%) |
33
+ | 4-bit AWQ (rejected) | 18.6% |
34
+
35
+ Additional checks: defect rate on par with bf16 (broken chars: 0), Korean particle-corruption scan at bf16 level, side-by-side human reading indistinguishable from bf16.
36
+
37
+ ## Usage (vLLM)
38
+
39
+ ```bash
40
+ vllm serve shleekr/gemma4-expAF-ckpt486-w8a16 --max-model-len 49152 --dtype bfloat16
41
+ ```
42
+
43
+ Includes the corrected `chat_template.jinja` (16,934 chars — with multi-turn continuation handling; older exports shipped a stale 12,045-char template).
44
+
45
+ Sampling used in validation: `temperature=0.8, top_p=0.9`.