verilog-qwen2.5-coder-7b-v9-auto-distilled-direct-lora

LoRA/PEFT adapter for Verilog RTL code generation, trained from Qwen/Qwen2.5-Coder-7B-Instruct through the project adapter chain.

Intended use

Load base model Qwen/Qwen2.5-Coder-7B-Instruct and this adapter with PEFT. Prompt for complete synthesizable Verilog. This repo contains adapter files plus tokenizer/config dependencies needed for PEFT loading.

Local VerilogEval v2 spec-to-RTL direct result

• Compile: 135/156 = 86.54%
• Functional pass: 67/156 = 42.95%

Best direct first-pass adapter in this experiment series.

Caveat

This adapter is benchmark-targeted. Training data includes VerilogEval-distilled passing outputs and verified/retention data. Do not present these numbers as clean zero-shot leaderboard results.

Training summary

• Base adapter input: v8 combined verified direct (for v9); v9 auto distilled direct (for v10)
• Dataset style: direct spec -> verified passing Verilog code
• LoRA r/alpha: 16/32
• LR: 3e-7
• Epochs: 1
• Checkpointing: enabled every 25 steps during training

Loading sketch

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = "Qwen/Qwen2.5-Coder-7B-Instruct"
adapter = "Pablo-Flores-Mollinedo/verilog-qwen2.5-coder-7b-v9-auto-distilled-direct-lora"

tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter)