---
license: apache-2.0
language: code
tags:
  - gemma
  - qlora
  - circuit-synthesis
  - verilog
  - llm
  - electronic-design-automation
  - peft
  - google-colab
model-index:
  - name: veriforge-gemma-2b-it
    results: []
---

# Veriforge-Gemma-2B-IT 🔧

**`veriforge-gemma-2b-it`** is a QLoRA-fine-tuned version of [`google/gemma-2b-it`](https://huggingface.co/google/gemma-2b-it) that specializes in prompt-based circuit synthesis for digital logic design, specifically in Verilog HDL.

## 🚀 Model Description

- **Base Model**: [`google/gemma-2b-it`](https://huggingface.co/google/gemma-2b-it)
- **Fine-tuned By**: [louijiec](https://huggingface.co/louijiec)
- **Method**: QLoRA using PEFT and bitsandbytes
- **Data**: 500 simulated Verilog gate examples (AND, OR, NAND, etc.)
- **Platform**: Google Colab

## 🧐 Example Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "louijiec/veriforge-gemma-2b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "### Prompt:\nWrite Verilog code for a 3-input XOR gate.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 🧪 Sample Output

```verilog
module nand_3_input (output y, input a0, a1, a2);
  assign y = ~(a0 & a1 & a2);
endmodule
```

## 📚 Training Details

- LoRA rank: 8
- Bits: 4-bit (QLoRA)
- Max tokens: 512
- Optimizer: AdamW, FP16
- Epochs: 10
- Batch Size: 2
- Gradient Accumulation: 4
- Logging Steps: 10

## 📌 Citations

- Gemma by Google: https://huggingface.co/google/gemma-2b-it
- QLoRA: https://arxiv.org/abs/2305.14314
- PEFT: https://github.com/huggingface/peft

## ⚠️ Limitations

- Trained only on simple gates
- No memory/state logic (flip-flops, FSMs, etc.)
- No formal verification or testbench evaluation

## 💪 Future Work

- Add support for more circuit components (MUX, ALU)
- Formal testbench generation
- Build EDA pipeline integrations