---
model_name: Connect4 QLoRA Adapter for Qwen3-0.6B-Base
version: 2.0.0
library_name: peft
base_model: Qwen/Qwen3-0.6B-Base
license: mit
tags:
  - lora
  - qlora
  - bitsandbytes
  - transformers
  - connect4
  - game-playing
  - causal-lm
datasets:
  - private/connect4-selfplay
pipeline_tag: text-generation
inference: false
language:
  - en
quantization_config:
  load_in_4bit: true
  bnb_4bit_quant_type: nf4
  bnb_4bit_compute_dtype: float16
---

# Connect 4 QLoRA Adapter for Qwen/Qwen3-0.6B-Base

## Model Summary

This repository distributes a QLoRA adapter trained to steer the **Qwen/Qwen3-0.6B-Base** model toward Connect Four next-move prediction and short-form move generation. Prompts encode game history as concatenated column indices (`0`–`6`) along with the starter and side-to-move context, allowing the adapter to focus on legal column selection. The weights are stored separately from the base checkpoint; load or merge them into the matching base revision before running inference. Runs on a single consumer GPU with 4 GB of VRAM in 4bit mode. 

This updated version includes a larger training set and automatically selected the checkpoint with the lowest evaluation loss for improved reliability and performance.

## How to Use

### Load for inference with PEFT

```python
from pathlib import Path

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL = "Qwen/Qwen3-0.6B-Base"
ADAPTER_PATH = "RenaudGaudron/Qwen3-0.6B-Connect4"

model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
    torch_dtype=torch.float16,
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)
model.eval()

PROMPT_TEMPLATE = (
    "Connect Four is played on a 7-column by 6-row grid. Players alternate"
    " dropping discs that stack upwards, and full columns cannot be used."
    "\nGame starter: Player {starter}"
    "\nPlayer to move: Player {current_player}"
    "\nMoves so far: {history}"
    "\nSelect a legal column (0-6) for Player {current_player} and avoid full columns."
    "\nAnswer with a single digit representing the column index."
    "\nResponse:"
)

example_prompt = PROMPT_TEMPLATE.format(
    starter=1,
    current_player=2,
    history="32344553",
)
inputs = tokenizer(example_prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=2,
        do_sample=False,
    )
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### Merge adapters into the base (optional)

```python
from pathlib import Path

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM

BASE_MODEL = "Qwen/Qwen3-0.6B-Base"
ADAPTER_PATH = "RenaudGaudron/Qwen3-0.6B-Connect4"
OUTPUT_DIR = Path("./merged-connect4-qwen3-0.6b")

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_4bit=False,
)
model = PeftModel.from_pretrained(base_model, ADAPTER_PATH)
merged = model.merge_and_unload()
merged.save_pretrained(OUTPUT_DIR)
model.tokenizer.save_pretrained(OUTPUT_DIR)

print(f"Merged model saved to {OUTPUT_DIR}. Review the base model license before redistribution.")
```

> **Prompt format.** Training examples follow the template above. The `Moves so far` string serialises column indices (`0`–`6`) without separators so column legality remains reconstructible. Use `none` when the board is empty, and always align the move history with the declared starter to avoid illegal column suggestions. Responses remain single digits; deterministic decoding or constrained vocabularies help preserve the format.

## Training Details

### LoRA configuration

* Method: QLoRA with a frozen 4-bit base model.
* Rank (`r`): 128.
* Scaling (`lora_alpha`): 256.
* Dropout (`lora_dropout`): 0.1.
* Target modules: attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`) and MLP projections (`gate_proj`, `up_proj`, `down_proj`).
* Adapter bias: disabled; only rank update matrices are trainable.

QLoRA keeps the dense Qwen3 backbone quantised to 4-bit NF4 while learning a lightweight LoRA stack initialised from the resumed checkpoint. The adapters stay in float32 while the quantised backbone executes in 4-bit, keeping peak VRAM well under 8 GB on the training GPU.

### Optimisation setup

* Optimiser: `adamw_torch_fused` with β₂ = 0.98 and ε = 1e-6.
* Learning rate: 5e-7 with `constant_with_warmup` scheduling and a warmup ratio of 5 %.
* Weight decay: 0.0.
* Gradient accumulation: 8 steps with per-device batch size 8 → effective batch size 64 sequences.
* Max gradient norm: 25.0 with clipping applied every optimisation step.
* Label smoothing: disabled.
* Attention backend: PyTorch SDPA with math kernel fallback; flash and memory-efficient kernels were unavailable on the training GPU.
* Length-aware sampling: 64-bucket sampler enabled to reduce padding skew during both training and evaluation.

### Precision and memory

* Base model loaded in 4-bit NF4 with double quantisation; LoRA weights stored in float32.
* Computation dtype: BF16 with TF32 matmuls enabled; FP16 disabled.
* Gradient checkpointing: disabled.

### Data

* Dataset source: private self-play Connect Four rollouts across three CSV files.
* Aggregated examples: 114 363 generated prompts (102 926 train / 11 437 validation) from 4 200 games.
* Per-file caps: 100 level-0 games (two players making random legal moves), 4 000 games played between Minimax agents of depth 8, and 100 mixed-level games (depth 4 vs 8). Each game is expanded into multiple move-prefix supervision sequences for training.
* Minimum move threshold: unset (`min_moves=0`) so every legal position contributes.
* Validation split: 10 % stratified after shuffle.

### Training run

* Epochs: 5 planned
* Best observed validation loss: **0.7196** at step 50 000 (saved as the published checkpoint).
* Training log span: ~60h with periodic evaluations every 1 000 steps.
* Hardware: single-process Accelerate session on a Windows workstation loading bitsandbytes CUDA 12.4 bindings and operating on a CUDA device.
* Seed: 42 across Python, NumPy, and PyTorch. Dataloader workers (10) and CUDA kernels may still introduce nondeterminism.

## Evaluation

Evaluation reuses the training prompt template with `eval_max_seq_length=128` truncation. The primary metric is token-level cross-entropy (reported as loss) measuring next-move prediction quality.

* Published checkpoint validation loss: **0.7196** (best checkpoint at step 50 000).

Loss values indicate that the adapter reliably tracks legal play patterns after the expanded curriculum while remaining sensitive to decoding constraints near terminal states.

Stability controls mirrored training: gradient clipping at norm 25.0, SDPA math kernel fallback, and warmup scheduling. No extra label smoothing or dropout beyond the LoRA stack was introduced.

## Special Tokens

The adapter ships without introducing new tokens. The bundled tokenizer metadata mirrors the base checkpoint, reusing `<|endoftext|>` as both EOS and padding alongside the stock Qwen multimodal specials.

## Intended Use & Limitations

**Intended use.** 

Pair the adapter with **Qwen/Qwen3-0.6B-Base** for Connect Four next-move suggestion or short move-sequence generation. Prompts should follow the documented template for reliable legality.

**Limitations.**

* The compact 0.6B backbone cannot guarantee optimal play in deep tactical lines; illegal or low-quality moves remain possible, especially near endgame scenarios.
* Outputs are restricted to single-digit column indices; free-form chat or multi-turn dialogue falls outside the training distribution.
* The adapter does not include safety layers, toxicity filtering, or alignment for open-domain generation. Avoid deploying it in user-facing production systems.
* Quality still depends on accurate, complete move histories and deterministic decoding (e.g., `do_sample=False` or constrained vocab sampling).

## Compatibility

The adapter was trained and validated with the following stack:

* `transformers` ≥ 4.39.0
* `peft` ≥ 0.8.2
* `bitsandbytes` ≥ 0.43.0
* `torch` ≥ 2.1 (CUDA 12.4 build per bitsandbytes log)
* `accelerate` ≥ 0.25.0

Windows-based CUDA 12.4 bindings powered the training environment. On Linux or alternate CUDA releases, Accelerate falls back to available SDPA kernels; if flash or memory-efficient kernels are missing (as during training), PyTorch defaults to the math implementation. Always load the same **Qwen/Qwen3-0.6B-Base** revision used during fine-tuning to avoid key mismatches.

## Reproducibility & Seeds

The global seed remained **42** across Python, NumPy, and PyTorch. Despite deterministic settings, CUDA kernel scheduling, dataloader worker ordering, and filesystem timing can introduce minor nondeterminism. 

## Changelog

* **2025-11-04** – Expanded dataset fine-tune with automated best-checkpoint capture.
* **2025-10-25** – Initial public adapter release.

## License

The adapter is released under the **MIT License**. The base model, **Qwen/Qwen3-0.6B-Base**, ships under Apache 2.0. Please ensure that downstream usage respects both licenses before merging, redistributing, or further fine-tuning.

## Citations

* Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv:2106.09685.
* Dettmers et al., “QLoRA: Efficient Finetuning of Quantized LLMs,” arXiv:2305.14314.
* Wolf et al., “HuggingFace's Transformers: State-of-the-Art Natural Language Processing,” arXiv:1910.03771.