qwen35-27b-ancient-rl-r32-step100

This repository contains the full merged bfloat16 model for the rank-32 GRPO checkpoint at step 100. It is not an adapter-only upload: the LoRA adapter has been merged into the Qwen3.5-27B ancient-language stage-2 base model, and the repository includes the safetensors shards plus tokenizer/processor/chat-template files needed for direct transformers loading.

Adapter-only sibling: MooreMuaMu/qwen35-27b-ancient-rl-r32-step100-lora.

Intended Use

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "MooreMuaMu/qwen35-27b-ancient-rl-r32-step100"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Training Source

  • Base model: /blob/models/qwen35-27b-ancient-stage2/checkpoint-227-merged
  • LoRA checkpoint: /blob/experiments/ancient-swift-rl-stage2/qwen35_27b_ckpt227_uy_mn_bo_ta_full_r32_lr1e4/v0-20260811-070520/checkpoint-100
  • LoRA rank/alpha: 32/32
  • Merge dtype: bfloat16
  • Serialization: safetensors, 5GB max shard size

Evaluation Summary

Evaluation used the independent six-bucket held-out subset: 300 prompts, 50 per bucket, thinking enabled, temperature 0.7, top-p 0.95, n=1, seed 42. Deltas are prompt-paired against the same base-model generations where applicable. Bootstrap confidence intervals use 10,000 paired samples.

Overall semantic metrics:

Metric Base Step 100 Delta 95% CI
Exact 0.3133 0.3400 +0.0267 [-0.0167, +0.0733]
Char-F1 0.5499 0.5579 +0.0080 [-0.0328, +0.0495]
BERTScore-F1 0.7638 0.7657 +0.0019 [-0.0309, +0.0348]
Semantic composite 0.5915 0.6015 +0.0100 [-0.0260, +0.0473]
Has answer 0.9533 0.9467 -0.0067 [-0.0400, +0.0267]

SacreBLEU2 answer-extraction metrics use sacrebleu.metrics.BLEU(tokenize="zh", max_ngram_order=2, smooth_method="exp", effective_order=True) after extracting text inside <ANS>...</ANS> and discarding any prefix before </think>.

Metric Base Step 100 Delta
Corpus BLEU-2 (%) 34.30 39.86 +5.56 pp
Sentence BLEU-2 mean (%) 46.12 48.10 +1.98 pp
Nonempty extracted answer 95.0% 94.7% -0.3 pp

Bucket-level SacreBLEU2:

Bucket Corpus BLEU-2 base -> step100 Delta Sentence BLEU-2 base -> step100 Delta Step100 nonempty
Tibetan annotation 59.14 -> 56.73 -2.41 pp 65.40 -> 66.82 +1.42 pp 94.0%
Tibetan translation 15.68 -> 18.15 +2.47 pp 16.43 -> 16.00 -0.43 pp 94.0%
Traditional Mongolian annotation 53.65 -> 70.42 +16.77 pp 50.84 -> 56.84 +6.01 pp 92.0%
Traditional Mongolian translation 25.52 -> 26.15 +0.63 pp 24.41 -> 29.26 +4.85 pp 100.0%
Uyghur annotation 85.13 -> 90.75 +5.62 pp 87.89 -> 87.26 -0.63 pp 88.0%
Uyghur translation 34.55 -> 36.67 +2.12 pp 31.77 -> 32.41 +0.64 pp 100.0%

Decision note: step 100 is an exploratory checkpoint. Its aggregate semantic improvement is small and the confidence interval crosses zero, although SacreBLEU2 corpus score improves, mainly from Traditional Mongolian annotation.

Downloads last month
39
Safetensors
Model size
27B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support