---
language:
- en
license: mit
library_name: peft
base_model: LiquidAI/LFM2.5-350M
tags:
- lora
- finetuned
- lfm
- code-generation
- python
- liquid-ai
- small-model
- on-device
pipeline_tag: text-generation
model-index:
- name: LFM2.5-350M-code_gen-finetuned
  results: []
---

# LFM2.5-350M-code_gen-finetuned

Fine-tuned version of **LiquidAI/LFM2.5-350M** for **Python code generation from natural language instructions**.

Generates clean, PEP 8 compliant Python code with docstrings and error handling.

## Model Details

| Attribute | Value |
|---|---|
| Base Model | LiquidAI/LFM2.5-350M |
| Parameters | 350M (full), ~2M trainable (LoRA) |
| Architecture | Hybrid (10 LIV convolution + 6 GQA blocks) |
| Context Length | 32,768 tokens |
| Fine-tuning Method | QLoRA (4-bit NF4) |
| LoRA Rank | 16 |
| LoRA Alpha | 32 |
| Dataset | TokenBender/code_instructions_122k_alpaca_style |
| Epochs | 3 |
| Learning Rate | 2e-4 |
| Optimizer | paged_adamw_32bit |

## Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-350M", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "LiquidAI/LFM2.5-350M",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "rohitnagareddy/LFM2.5-350M-code_gen-finetuned")

# Generate code
messages = [
    {"role": "system", "content": "You are an expert Python programmer. Write clean, efficient, and well-documented code."},
    {"role": "user", "content": "Write a function that finds the longest palindromic substring"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, top_p=0.9, repetition_penalty=1.3, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
```

## Training Details

- **LoRA Target Modules:** q_proj, k_proj, v_proj, o_proj
- **LoRA Dropout:** 0.05
- **Gradient Accumulation:** 4 steps
- **Scheduler:** Cosine with 3% warmup
- **Max Gradient Norm:** 1.0
- **Weight Decay:** 0.01
- **Quantization:** 4-bit NF4 with double quantization

## Intended Use

Python code generation from natural language descriptions. Useful for:
- Quick function prototyping
- Code completion assistance
- Learning Python patterns
- Boilerplate generation

## Limitations

- 350M parameters — generates simple-to-medium complexity code
- May produce syntactically correct but logically flawed solutions for complex algorithms
- Not a replacement for human code review
- Knowledge cutoff: mid-2024

## Author

Fine-tuned by [rohitnagareddy](https://huggingface.co/rohitnagareddy)

## License

MIT License