| --- |
| license: apache-2.0 |
| base_model: unsloth/Qwen3.5-4B-Base |
| tags: [tajik, qwen3.5, continual-pretraining, cpt, merged, stacked-lora] |
| language: [tg] |
| --- |
| |
| # Qwen3.5-4B Tajik — CPT epoch-2 (stacked), merged |
|
|
| Continual-pretraining (CPT) adaptation of **Qwen3.5-4B-Base** to **Tajik** (тоҷикӣ), **2 stacked epochs**, |
| with the LoRA adapters merged into the base weights (BF16). This is the **stacked-LoRA** chain: |
| epoch-2 is a *fresh* rank-64 LoRA trained on top of the merged **base+e1** checkpoint |
| (`Tohirju/qwen35-4b-tajik-cpt-e1-merged`), then merged in — i.e. `base + e1 + e2`. |
|
|
| - **What it is:** Qwen3.5-4B-Base after 2 epochs of Tajik continual pre-training (merge-restacked per epoch). |
| - **Use:** a stronger Tajik base for generation and as the starting point for the next CPT epoch or for SFT. |
| - **Tajik MCQ (Zehnlab, full ~18,970 Q, loglikelihood):** see the eval repo `Tohirju/qwen35-tajik-eval-results`. |
| For reference, base=44.7%, base+e1=48.9%. |
| - **Note:** this is a base/CPT model — it *completes* text, it does not follow chat instructions (that is the SFT stage). |
| - Merge verified by logit parity against the e2 adapter (max|Δlogit| < 1e0). |
| Components: `unsloth/Qwen3.5-4B-Base` + `Tohirju/qwen35-4b-tajik-cpt-e1-merged` + `Tohirju/qwen35-4b-tajik-cpt-stacked-e2`. |
|
|
| Gated: download requires manual approval. Released by Saidzoda Lab / Saidzoda Engineering Company. |
|
|
