ai-talent-finder-backend / FINETUNE_README.md
ilyass yani
Deploiement backend dans HF Spaces
9df97a2
|
Raw
History Blame
829 Bytes
# GPU Finetuning (LoRA / QLoRA)
Steps to run a full finetune on GPU:
1. Prepare an `accelerate` config (see `backend/accelerate_config_demo.yaml`).
2. Ensure the venv has `transformers`, `peft`, `accelerate`, `bitsandbytes`, and `datasets` installed. Example:
```
.venv/bin/pip install -r backend/requirements-extras.txt
.venv/bin/pip install peft bitsandbytes accelerate -U
```
3. Run the helper script in dry-run first, then without `--dry-run`:
```
./backend/scripts/run_finetune_gpu.sh --model mistralai/Mistral-7B-Instruct-v0.1 --data data/finetune_sample.jsonl --out models/mistral_finetuned --method qlora --epochs 3 --batch 4
```
Notes:
- Training large models requires appropriate GPU memory and CUDA drivers.
- When ready, remove `--dry-run` in `run_finetune_gpu.sh` or edit the command to perform the real run.