File size: 829 Bytes
9df97a2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # GPU Finetuning (LoRA / QLoRA)
Steps to run a full finetune on GPU:
1. Prepare an `accelerate` config (see `backend/accelerate_config_demo.yaml`).
2. Ensure the venv has `transformers`, `peft`, `accelerate`, `bitsandbytes`, and `datasets` installed. Example:
```
.venv/bin/pip install -r backend/requirements-extras.txt
.venv/bin/pip install peft bitsandbytes accelerate -U
```
3. Run the helper script in dry-run first, then without `--dry-run`:
```
./backend/scripts/run_finetune_gpu.sh --model mistralai/Mistral-7B-Instruct-v0.1 --data data/finetune_sample.jsonl --out models/mistral_finetuned --method qlora --epochs 3 --batch 4
```
Notes:
- Training large models requires appropriate GPU memory and CUDA drivers.
- When ready, remove `--dry-run` in `run_finetune_gpu.sh` or edit the command to perform the real run.
|