Upload Qwen3-4B P300 text correction LoRA checkpoints

LICENSE.md

@@ -0,0 +1,115 @@
+# License and mixed-source terms
+
+This repository uses the Hugging Face metadata value `license: other` because the
+adapter was trained from sources offered under different licenses. It contains
+PEFT LoRA adapter weights and documentation, but does not redistribute the base
+Qwen model or any training dataset rows.
+
+| Source | License | Revision |
+|---|---|---|
+| [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) | Apache-2.0 | `cdbee75f17c01a7cc42f958dc650907174af0554` |
+| [RuSentiment](https://github.com/strawberrypie/rusentiment) | CC BY-NC-SA 4.0 | `7f2afa11a9483f5251cd5156ac848d098b502be0` |
+| [Dialogs](https://huggingface.co/datasets/langswap/dialogs-ru-emotional-conversations) | OpenRAIL | `e25ba617b2b56bd1dbf255d3905c51bd8da3d31f` |
+
+Subject to all applicable upstream terms, the project author's copyright
+interest, if any, in the adapter weights and model documentation is made
+available under
+[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
+No additional permission for commercial use is granted.
+
+To the extent that the RuSentiment license applies to the adapter weights or
+their use, recipients must comply with its attribution, non-commercial, and
+share-alike conditions. The legal status of learned adapter weights as
+derivative material and the compatibility of the upstream licenses are not
+asserted. Recipients are responsible for determining whether their intended use
+is permitted.
+
+The exact Dialogs OpenRAIL license supplied with the training revision is
+reproduced below because its distribution terms require the license and use
+restrictions to flow down to recipients.
+
+---
+
+# Dialogs — OpenRAIL License
+
+This dataset ("the Dataset"), and any models or other works trained on or derived
+from it together with their outputs ("Derivatives"), are licensed under an **Open
+& Responsible AI License (OpenRAIL)**, following the BigScience / CreativeML
+OpenRAIL-M template (see https://www.licenses.ai/ and
+https://huggingface.co/blog/open_rail). By accessing or using the Dataset you
+accept these terms.
+
+**Licensor:** Ilya Shigabeev and Ilya Latyshev (Langswap).
+
+## 1. Grant of rights
+
+Subject to the use restrictions in Section 2, the Licensor grants you a
+perpetual, worldwide, non-exclusive, royalty-free, irrevocable license to access,
+use, reproduce, modify, publicly perform, distribute, and redistribute the
+Dataset and Derivatives, for any purpose, **including commercial purposes**.
+
+The Licensor claims no rights over Derivatives you create or outputs you
+generate, provided your use complies with Section 2.
+
+## 2. Use Restrictions (Attachment A)
+
+You agree not to use the Dataset or Derivatives:
+
+1. In any way that violates any applicable national, federal, state, local or
+   international law or regulation;
+2. For the purpose of exploiting, harming or attempting to exploit or harm
+   minors in any way;
+3. To generate or disseminate verifiably false information and/or content with
+   the purpose of harming others;
+4. To generate or disseminate personal identifiable information that can be
+   used to harm an individual;
+5. To defame, disparage or otherwise harass others;
+6. For fully automated decision making that adversely impacts an individual's
+   legal rights or otherwise creates or modifies a binding, enforceable
+   obligation;
+7. For any use intended to or which has the effect of discriminating against or
+   harming individuals or groups based on online or offline social behavior or
+   known or predicted personal or personality characteristics;
+8. To exploit any of the vulnerabilities of a specific group of persons based
+   on their age, social, physical or mental characteristics, in order to
+   materially distort the behavior of a person pertaining to that group in a
+   manner that causes or is likely to cause that person or another person
+   physical or psychological harm;
+9. For any use intended to or which has the effect of discriminating against
+   individuals or groups based on legally protected characteristics or
+   categories;
+10. To provide medical advice and medical results interpretation;
+11. To generate or disseminate information for the purpose to be used for
+    administration of justice, law enforcement, immigration or asylum
+    processes, such as predicting an individual will commit fraud/crime
+    commitment (e.g. by text profiling, drawing causal relationships between
+    assertions made in documents, indiscriminate and arbitrarily-targeted use).
+
+## 3. Distribution and flow-down
+
+If you distribute the Dataset or any Derivative, you must (a) provide a copy of
+this license, including Section 2, to all recipients, and (b) require recipients
+to comply with Section 2. You may add your own terms for your Derivatives
+provided they do not conflict with or weaken these Use Restrictions.
+
+## 4. Disclaimer of warranty and liability
+
+The Dataset is provided "AS IS", without warranties of any kind, express or
+implied. To the maximum extent permitted by law, the Licensor is not liable for
+any claim, damages, or liability arising from use of the Dataset or Derivatives.
+
+---
+
+*This is an OpenRAIL license (the BigScience/CreativeML OpenRAIL-M template, with
+its standard Attachment A use restrictions, applied to a dataset). On Hugging
+Face this dataset is tagged `license: openrail`. The Dataset's voice actors gave
+written informed consent for open release and lawful use, including commercial
+use.*
+
+## Adapter disclaimer
+
+THE ADAPTER WEIGHTS AND DOCUMENTATION ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT.
+IN NO EVENT SHALL THE PROJECT AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER
+LIABILITY ARISING FROM OR RELATED TO THE ADAPTER OR ITS USE.

README.md

@@ -1,5 +1,150 @@
 ---
+base_model: Qwen/Qwen3-4B-Instruct-2507
+base_model_relation: adapter
+library_name: peft
+pipeline_tag: text-generation
+language:
+  - ru
+tags:
+  - qwen3
+  - peft
+  - lora
+  - qlora
+  - text-correction
+  - p300-speller
+datasets:
+  - strawberrypie/rusentiment
+  - langswap/dialogs-ru-emotional-conversations
 license: other
-license_name: mixed
-license_link: LICENSE
+license_name: mixed-source-cc-by-nc-sa-4.0-and-openrail
+license_link: LICENSE.md
 ---
+
+# Qwen3-4B P300 Text Correction LoRA
+
+Two PEFT LoRA checkpoints fine-tuned from
+[`Qwen/Qwen3-4B-Instruct-2507`](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
+for restoring short Russian phrases affected by synthetic character-level
+substitution, adjacent duplication, and deletion noise.
+
+The artificial corruption imitates character errors that may occur in decoded
+P300-speller text.
+
+## Checkpoints
+
+| Hub path | Epoch | Step | Status |
+|---|---:|---:|---|
+| repository root | 1 | 780 | selected default |
+| `checkpoint-1560/` | 2 | 1560 | retained alternative |
+
+Checkpoint 780 had lower clean and mixed validation micro CER and fewer
+unnecessary clean modifications. Checkpoint 1560 is included for reproducibility,
+not because it performed better.
+
+## Quick start
+
+The tokenizer and chat template must be loaded from the pinned base model. To
+use checkpoint 1560, set `adapter_kwargs` to the commented alternative below.
+
+```python
+from pathlib import Path
+
+import torch
+from huggingface_hub import hf_hub_download
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base_id = "Qwen/Qwen3-4B-Instruct-2507"
+base_revision = "cdbee75f17c01a7cc42f958dc650907174af0554"
+adapter_id = "redche7/qwen3-4b-p300-text-correction-lora"
+
+tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_id,
+    revision=base_revision,
+    dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+adapter_kwargs = {}
+# adapter_kwargs = {"subfolder": "checkpoint-1560"}
+model = PeftModel.from_pretrained(base_model, adapter_id, **adapter_kwargs)
+model.eval()
+
+prompt_path = hf_hub_download(adapter_id, "prompt.txt")
+system_prompt = Path(prompt_path).read_text(encoding="utf-8")
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": "ПРИВЕТ, КАК ДИЛА?"},
+]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+).to(model.device)
+
+with torch.inference_mode():
+    generated = model.generate(**inputs, do_sample=False, max_new_tokens=160)
+
+prompt_length = inputs["input_ids"].shape[1]
+correction = tokenizer.decode(
+    generated[0, prompt_length:],
+    skip_special_tokens=True,
+).strip()
+print(correction)
+```
+
+## Training
+
+The adapters were trained with QLoRA on 22,698 deterministic
+prompt-completion records. Source labels and audio were not used as targets, and
+no training rows are distributed in this model repository.
+
+| Text source | Revision | Records | License |
+|---|---|---:|---|
+| [RuSentiment](https://github.com/strawberrypie/rusentiment) | `7f2afa11a9483f5251cd5156ac848d098b502be0` | 15,158 | CC BY-NC-SA 4.0 |
+| [Dialogs](https://huggingface.co/datasets/langswap/dialogs-ru-emotional-conversations) | `e25ba617b2b56bd1dbf255d3905c51bd8da3d31f` | 7,540 | OpenRAIL |
+
+Key settings were NF4 QLoRA with BF16 compute, LoRA
+`r=16`, `alpha=32`, dropout `0.05`, maximum sequence length 384, effective batch
+size 16, and learning rate `1e-4`. The full machine-independent summary is in
+[training_config.yaml](training_config.yaml).
+
+## Evaluation
+
+Both checkpoints were compared on the same 300 clean and 300 synthetically
+corrupted validation requests:
+
+| Checkpoint | Clean micro CER | Mixed micro CER | Unnecessary clean modifications |
+|---|---:|---:|---:|
+| checkpoint 780 | 1.9407% | 19.5066% | 54 / 300 |
+| checkpoint 1560 | 2.0616% | 19.6204% | 57 / 300 |
+
+The selected checkpoint 780 was then compared with the frozen BF16 base model:
+
+| Evaluation slice | Rows | Base micro CER | Adapter micro CER |
+|---|---:|---:|---:|
+| in-domain mixed | 1,000 | 22.4742% | 20.4710% |
+| external mixed | 2,000 | 22.5886% | 18.8385% |
+| external clean control | 2,000 | 1.6814% | 0.4886% |
+
+The external set contains 1,600 human Opusparcus-derived phrases and 400
+synthetic phrases. The clean control uses the original phrases, while the mixed
+condition applies the same artificial character-corruption protocol.
+
+On the external clean control, unnecessary modifications decreased from 24.35%
+to 5.85%. These results apply only to the frozen artificial-corruption protocol,
+prompt, deterministic decoding, Russian text sources, and evaluated phrase
+lengths.
+
+## License
+
+The training sources use different licenses, so the repository is marked
+`license: other` rather than Apache-2.0. No commercial-use clearance or claim of
+cross-license compatibility is provided. See [LICENSE.md](LICENSE.md) for the
+applicable CC BY-NC-SA 4.0 and OpenRAIL terms.
+
+Exact adapter hashes, sizes, checkpoint mapping, and verification status are in
+[release_manifest.json](release_manifest.json).

adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef9507fe17f03a1f725c322aaafcc3d262953f8bff4137e048b299f32e92987f
+size 66127776

checkpoint-1560/adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1560/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23306685a0165482806860e5d63bfc9a5388faae4e55362fee0824a628c1d9db
+size 66127776

prompt.txt

@@ -0,0 +1,13 @@
+Recover the most likely original Russian phrase from the final user message.
+
+The input is either unchanged or was corrupted only by independent character-level noise.
+Individual characters may have been substituted, duplicated directly beside themselves, or deleted completely.
+Multiple errors may occur in the same phrase.
+The corruption does not reorder the surviving characters.
+
+Restore the exact original wording and punctuation.
+Do not paraphrase, replace words with synonyms, or rewrite the phrase.
+Preserve informal language, slang, and source errors not caused by the described corruption.
+
+Return exactly one corrected Russian phrase and nothing else.
+If the phrase is already correct, return it unchanged.

release_manifest.json

@@ -0,0 +1,60 @@
+{
+  "schema_version": "huggingface_adapter_release.v1",
+  "base_model": {
+    "model_id": "Qwen/Qwen3-4B-Instruct-2507",
+    "revision": "cdbee75f17c01a7cc42f958dc650907174af0554"
+  },
+  "default_checkpoint": {
+    "epoch": 1,
+    "global_step": 780,
+    "path": ".",
+    "selected": true,
+    "adapter_size_bytes": 66127776,
+    "adapter_sha256": "ef9507fe17f03a1f725c322aaafcc3d262953f8bff4137e048b299f32e92987f"
+  },
+  "alternative_checkpoints": [
+    {
+      "epoch": 2,
+      "global_step": 1560,
+      "path": "checkpoint-1560",
+      "selected": false,
+      "adapter_size_bytes": 66127776,
+      "adapter_sha256": "23306685a0165482806860e5d63bfc9a5388faae4e55362fee0824a628c1d9db"
+    }
+  ],
+  "training": {
+    "record_count": 22698,
+    "token_count": 4784492,
+    "source_records": {
+      "strawberrypie/rusentiment": 15158,
+      "langswap/dialogs-ru-emotional-conversations": 7540
+    }
+  },
+  "license": {
+    "huggingface_id": "other",
+    "name": "mixed-source-cc-by-nc-sa-4.0-and-openrail",
+    "notice_path": "LICENSE.md"
+  },
+  "verification": {
+    "verified_on": "2026-07-23",
+    "model_card_metadata_local_parse": "passed",
+    "peft_config_load": "passed",
+    "safetensors_hash_tensor_and_finite_check": "passed",
+    "offline_base_and_adapter_load_smoke": {
+      "checkpoint-780": "passed",
+      "checkpoint-1560": "passed"
+    },
+    "huggingface_server_yaml_validation": "not_run_before_upload"
+  },
+  "excluded_checkpoint_files": [
+    "README.md",
+    "chat_template.jinja",
+    "optimizer.pt",
+    "rng_state.pth",
+    "scheduler.pt",
+    "tokenizer.json",
+    "tokenizer_config.json",
+    "trainer_state.json",
+    "training_args.bin"
+  ]
+}

training_config.yaml

@@ -0,0 +1,38 @@
+base_model: Qwen/Qwen3-4B-Instruct-2507
+base_revision: cdbee75f17c01a7cc42f958dc650907174af0554
+seed: 20260720
+train_records: 22698
+train_tokens: 4784492
+epochs: 2
+max_length: 384
+per_device_train_batch_size: 2
+gradient_accumulation_steps: 8
+effective_batch_size: 16
+learning_rate: 1.0e-4
+warmup_ratio: 0.03
+optimizer: paged_adamw_8bit
+lr_scheduler_type: linear
+max_grad_norm: 1.0
+weight_decay: 0.0
+completion_only_loss: true
+packing: true
+packing_strategy: wrapped
+gradient_checkpointing: true
+quantization:
+  load_in_4bit: true
+  quant_type: nf4
+  compute_dtype: bfloat16
+  double_quantization: true
+lora:
+  rank: 16
+  alpha: 32
+  dropout: 0.05
+  bias: none
+  target_modules: all-linear
+software:
+  torch: 2.11.0+cu128
+  cuda: "12.8"
+  transformers: 5.14.1
+  peft: 0.19.1
+  trl: 1.8.0
+  bitsandbytes: 0.49.2