---
license: apache-2.0
language:
  - en
base_model: Qwen/Qwen3-4B-Instruct-2507
pipeline_tag: text-generation
library_name: peft
tags:
  - lora
  - peft
  - qwen3
  - floorplan
  - hypergraph
---

# HypergraphFormer

Link to paper: https://arxiv.org/abs/2605.18932
LoRA adapters fine-tuning **Qwen/Qwen3-4B-Instruct-2507** for hypergraph-based
floorplan generation. The repo contains several adapters trained on
different dataset sizes.

## Checkpoints

| Subfolder | Train samples | Step |
|---|---|---|
| `qwen_hypergraphformer_1000_samples/checkpoint-240`  | 1,000  | 240  |
| `qwen_hypergraphformer_5000_samples/checkpoint-750`  | 5,000  | 750  |
| `qwen_hypergraphformer_10000_samples/checkpoint-1500`| 10,000 | 1500 |
| `qwen_hypergraphformer_25000_samples/checkpoint-3900`| 25,000 | 3900 |
| `qwen_hypergraphformer/checkpoint-8700`              | full   | 8700 |

## LoRA configuration

- Rank `r = 64`, `lora_alpha = 128`, `lora_dropout = 0.1`
- Target modules: `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
- Task: `CAUSAL_LM`

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-4B-Instruct-2507"
repo_id = "NikitaKlimenko/HypergraphFormer"
subfolder = "qwen_hypergraphformer_25000_samples/checkpoint-3900"

tok = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder)
base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, repo_id, subfolder=subfolder)