---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-32B-Instruct
tags:
- qwen2.5
- code
- dbt
- merged
- unsloth
library_name: transformers
---

# Qwen2.5-Coder-32B DBT Fine-tuned Model (Merged)

This is a merged model created from Qwen/Qwen2.5-Coder-32B-Instruct fine-tuned with LoRA for DBT (Data Build Tool) code generation tasks.

## Model Details

- **Base Model**: Qwen/Qwen2.5-Coder-32B-Instruct
- **Fine-tuning Method**: LoRA (merged into full weights)
- **Task**: Code generation for DBT SQL models
- **Training Framework**: Unsloth + Transformers

## Usage with VLLM

```bash
python -m vllm.entrypoints.openai.api_server \
    --model neutrino12/tensorstax-sft-unsloth-32b-lora32-3020 \
    --host 0.0.0.0 \
    --port 8000 \
    --max-model-len 32768 \
    --trust-remote-code
```

## Usage with Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "neutrino12/tensorstax-sft-unsloth-32b-lora32-3020",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    "neutrino12/tensorstax-sft-unsloth-32b-lora32-3020",
    trust_remote_code=True
)

# Generate text
inputs = tokenizer("Your prompt here", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
```

## Training Details

This model was fine-tuned using LoRA with the following configuration:
- LoRA Rank: 32
- LoRA Alpha: 64
- Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Learning Rate: 1e-5
- Sequence Length: 8000

The LoRA adapter was then merged back into the full model weights for easier deployment.