Upload 4 files (#1)

- Upload 4 files (6665e36a46a250377d4e8db5c9d0b1d7b046d0e1)

README.md

@@ -1,3 +1,214 @@
 ---
+model_name: Connect4 QLoRA Adapter for Qwen3-0.6B-Base
+version: 1.0.0
+library_name: peft
+base_model: Qwen/Qwen3-0.6B-Base
 license: mit
+tags:
+  - lora
+  - qlora
+  - bitsandbytes
+  - transformers
+  - connect4
+  - game-playing
+  - causal-lm
+datasets:
+  - private/connect4-selfplay
+pipeline_tag: text-generation
+inference: false
+language:
+  - en
+quantization_config:
+  load_in_4bit: true
+  bnb_4bit_quant_type: nf4
+  bnb_4bit_compute_dtype: float16
 ---
+
+# Connect 4 QLoRA Adapter for Qwen/Qwen3-0.6B-Base
+
+## Model Summary
+
+This repository distributes a QLoRA adapter trained to steer the **Qwen/Qwen3-0.6B-Base** model toward Connect Four next-move prediction and short-form move generation. Prompts encode game history as concatenated column indices (`0`–`6`) along with the starter and side-to-move context, allowing the adapter to focus on legal column selection. The weights are stored separately from the base checkpoint; load or merge them into the matching base revision before running inference. Runs on a single consumer GPU with 4 GB of VRAM in 4bit mode.
+
+## How to Use
+
+### Load for inference with PEFT
+
+```python
+from pathlib import Path
+
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+BASE_MODEL = "Qwen/Qwen3-0.6B-Base"
+ADAPTER_PATH = "RenaudGaudron/Qwen3-0.6B-Connect4"
+
+model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL,
+    device_map="auto",
+    torch_dtype=torch.float16,
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.float16,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+)
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
+model = PeftModel.from_pretrained(model, ADAPTER_PATH)
+model.eval()
+
+PROMPT_TEMPLATE = (
+    "Connect Four is played on a 7-column by 6-row grid. Players alternate"
+    " dropping discs that stack upwards, and full columns cannot be used."
+    "\nGame starter: Player {starter}"
+    "\nPlayer to move: Player {current_player}"
+    "\nMoves so far: {history}"
+    "\nSelect a legal column (0-6) for Player {current_player} and avoid full columns."
+    "\nAnswer with a single digit representing the column index."
+    "\nResponse:"
+)
+
+example_prompt = PROMPT_TEMPLATE.format(
+    starter=1,
+    current_player=2,
+    history="32344553",
+)
+inputs = tokenizer(example_prompt, return_tensors="pt").to(model.device)
+with torch.inference_mode():
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=2,
+        do_sample=False,
+    )
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Merge adapters into the base (optional)
+
+```python
+from pathlib import Path
+
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM
+
+BASE_MODEL = "Qwen/Qwen3-0.6B-Base"
+ADAPTER_PATH = "RenaudGaudron/Qwen3-0.6B-Connect4"
+OUTPUT_DIR = Path("./merged-connect4-qwen3-0.6b")
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL,
+    torch_dtype=torch.float16,
+    device_map="auto",
+    load_in_4bit=False,
+)
+model = PeftModel.from_pretrained(base_model, ADAPTER_PATH)
+merged = model.merge_and_unload()
+merged.save_pretrained(OUTPUT_DIR)
+model.tokenizer.save_pretrained(OUTPUT_DIR)
+
+print(f"Merged model saved to {OUTPUT_DIR}. Review the base model license before redistribution.")
+```
+
+> **Prompt format.** Every training example uses the template shown above. The `Moves so far` line encodes the move history as a contiguous string of digits where each character represents the column index (0–6) chosen at that ply. Use `none` when the position is empty, and keep the history aligned with the game starter so that column legality can be inferred correctly. The response should be exactly one digit; sampling or constrained decoding can help maintain this format.
+
+## Training Details
+
+### LoRA configuration
+
+* Method: QLoRA with a frozen 4-bit base model.
+* Rank (`r`): 128.
+* Scaling (`lora_alpha`): 256.
+* Dropout (`lora_dropout`): 0.1.
+* Target modules: attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`) and MLP projections (`gate_proj`, `up_proj`, `down_proj`).
+* Adapter bias: disabled; only rank update matrices are trainable.
+
+QLoRA keeps the dense Qwen3 backbone quantised to 4-bit NF4 while learning a lightweight set of low-rank matrices. This dramatically reduces memory pressure (the run fit comfortably in <10 GB of VRAM) and allows experimentation on modest hardware without sacrificing too much accuracy.
+
+### Optimisation setup
+
+* Optimiser: `adamw_torch_fused` with β₂ = 0.98 and ε = 1e-6.
+* Learning rate: 5e-7 with `constant_with_warmup` scheduling and a warmup ratio of 5 % (no fixed warmup steps beyond the ratio).
+* Weight decay: 0.0.
+* Gradient accumulation: 8 steps with per-device batch size 8 → effective batch size 64 sequences.
+* Max gradient norm: 25.0 with gradient clipping applied to stabilise updates.
+* Label smoothing: disabled.
+* Attention backend: PyTorch SDPA with math kernel fallback (flash and memory-efficient kernels were unavailable on the training GPU).
+
+### Precision and memory
+
+* Base model loaded in 4-bit NF4 with double quantisation; LoRA weights in float32.
+* Computation dtype: float16; BF16 enabled where supported (trainer configuration set `bf16=True`, `fp16=False`, `tf32=True`).
+* Gradient checkpointing: disabled.
+
+### Data
+
+* Dataset: private self-play Connect Four rollouts.
+* Move encoding: each game history is serialised as digits `0`–`6`, one per ply, reflecting the column placements from left (0) to right (6).
+* Total generated prompt-response pairs: 54 657 (49 191 train / 5 466 validation).
+* Minimum move threshold: 6 plies before emitting supervision to ensure non-trivial contexts.
+* Validation split: 10 % stratified by shuffle.
+
+### Training run
+
+* Epochs: 5 planned (converged by step 30 k of 30 745 total optimiser steps).
+* Total training runtime: ~26.7 h (96 205 s) with 0 out-of-memory retries.
+* Throughput: 2.56 samples/s, 0.32 steps/s.
+* Hardware: single-GPU Accelerate run (device 0, 4-bit load) on Windows; memory planner reserved 90 % of the device for model weights according to the log.
+* Seed: 42 (applied to Python, NumPy, and PyTorch). Standard dataloader shuffles and CUDA kernels may still introduce nondeterminism.
+
+## Evaluation
+
+Evaluation used the same prompt template and truncated each sequence to `eval_max_seq_length=128`. The primary metric is token-level cross-entropy (reported as loss), which acts as a proxy for perplexity over the next-move token.
+
+* Final checkpoint validation loss: **0.8570**.
+* Best observed validation loss during training: **0.7306** at step 30 000.
+
+These values indicate the adapter tracks legal play patterns and improves over the base model for move prediction, but decoding quality remains sensitive to sampling strategy and prompt fidelity. No additional metrics (win-rate or exact-match) were computed for this release.
+
+Stability controls included gradient clipping at a max norm of 25.0, continuous gradient-norm monitoring, and the warmup schedule above. No label smoothing or dropout beyond LoRA-specific dropout was used.
+
+## Special Tokens
+
+The adapter relies on the base tokenizer without introducing new tokens. The provided `special_tokens.json` simply reuses token ID `151643` for both padding and EOS with right-side padding. No custom BOS token or move separator tokens were added.
+
+## Intended Use & Limitations
+
+**Intended use.** Researchers and hobbyists experimenting with Connect Four agents can combine this adapter with Qwen/Qwen3-0.6B-Base to generate legal column suggestions. The model expects textual prompts that follow the documented template.
+
+**Limitations.**
+
+* The small 0.6 B parameter backbone limits long-horizon reasoning and nuanced board evaluation. Expect occasional illegal or weak moves, especially in edge cases near game end.
+* Outputs are single-digit column indices; free-form text prompts or multi-turn conversations fall outside the training distribution.
+* The adapter does not include safety layers, toxicity filtering, or alignment for open-domain generation. Avoid deploying it in user-facing production systems.
+* Quality depends heavily on providing complete, legal move histories and using deterministic decoding (`do_sample=False` or constrained vocab sampling).
+
+## Compatibility
+
+The adapter was trained and validated with the following stack:
+
+* `transformers` ≥ 4.39.0
+* `peft` ≥ 0.8.2
+* `bitsandbytes` ≥ 0.43.0
+* `torch` ≥ 2.1 (CUDA 12.4 compatible per bitsandbytes log)
+* `accelerate` ≥ 0.25.0
+
+The training environment ran on Windows with CUDA 12.4 bitsandbytes bindings. On Linux or other CUDA versions, Accelerate automatically falls back to the available SDPA kernels; if flash or memory-efficient SDPA kernels are missing (as during training), PyTorch will use the math implementation. Pin the same `Qwen/Qwen3-0.6B-Base` revision you fine-tune against to avoid key mismatches when loading the adapter.
+
+## Reproducibility & Seeds
+
+The training script fixed the global seed to **42** across Python, NumPy, and PyTorch. Nevertheless, sources of nondeterminism remain (CUDA kernels, dataloader worker order, and filesystem scheduling). For faithful reproduction, combine the published configuration with deterministic CUDA flags and ensure dataset shuffling uses the same seed.
+
+## Changelog
+
+* **2025-10-25** – Initial public adapter release.
+
+## License
+
+The adapter is released under the **MIT License**. The base model, **Qwen/Qwen3-0.6B-Base**, is distributed under the Apache License 2.0; ensure downstream usage complies with both the adapter’s MIT terms and the base model’s requirements before deploying, fine-tuning further, or redistributing merged checkpoints.
+
+## Citations
+
+* Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv:2106.09685.
+* Dettmers et al., “QLoRA: Efficient Finetuning of Quantized LLMs,” arXiv:2305.14314.
+* Wolf et al., “HuggingFace's Transformers: State-of-the-Art Natural Language Processing,” arXiv:1910.03771.

adapter_config.json

@@ -0,0 +1,42 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-0.6B-Base",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 256,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b66474a92f61223dcbd73507a84ea9d78a35e3808edbaf650e1706578d852eac
+size 161533944

special_tokens.json

@@ -0,0 +1,6 @@
+{
+  "pad_token_id": 151643,
+  "eos_token_id": 151643,
+  "bos_token_id": null,
+  "padding_side": "right"
+}