metadata
license: apache-2.0
base_model: unsloth/Qwen3.5-4B-Base
tags:
- tajik
- qwen3.5
- continual-pretraining
- cpt
- merged
- stacked-lora
language:
- tg
Qwen3.5-4B Tajik — CPT epoch-2 (stacked), merged
Continual-pretraining (CPT) adaptation of Qwen3.5-4B-Base to Tajik (тоҷикӣ), 2 stacked epochs,
with the LoRA adapters merged into the base weights (BF16). This is the stacked-LoRA chain:
epoch-2 is a fresh rank-64 LoRA trained on top of the merged base+e1 checkpoint
(Tohirju/qwen35-4b-tajik-cpt-e1-merged), then merged in — i.e. base + e1 + e2.
- What it is: Qwen3.5-4B-Base after 2 epochs of Tajik continual pre-training (merge-restacked per epoch).
- Use: a stronger Tajik base for generation and as the starting point for the next CPT epoch or for SFT.
- Tajik MCQ (Zehnlab, full ~18,970 Q, loglikelihood): see the eval repo
Tohirju/qwen35-tajik-eval-results. For reference, base=44.7%, base+e1=48.9%. - Note: this is a base/CPT model — it completes text, it does not follow chat instructions (that is the SFT stage).
- Merge verified by logit parity against the e2 adapter (max|Δlogit| < 1e0).
Components:
unsloth/Qwen3.5-4B-Base+Tohirju/qwen35-4b-tajik-cpt-e1-merged+Tohirju/qwen35-4b-tajik-cpt-stacked-e2.
Gated: download requires manual approval. Released by Saidzoda Lab / Saidzoda Engineering Company.