---
title: FAC-Synthesis Demo
emoji: 🧠
colorFrom: indigo
colorTo: red
sdk: streamlit
app_file: app.py
pinned: false
arxiv: 2602.10388
---

# FAC-Synthesis Demo

Interactive demo for "Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs"

## Quick Start

### Local Testing

```bash
cd demo
pip install -r requirements.txt
streamlit run app.py
```

Open http://localhost:8501 in your browser.

## Deploy to Public (Recommended: Streamlit Cloud)

### Step 1: Push to GitHub

```bash
cd demo
git init
git add .
git commit -m "Initial commit"
git remote add origin https://github.com/YOUR_USERNAME/fac-synthesis-demo.git
git branch -M main
git push -u origin main
```

### Step 2: Deploy on Streamlit Cloud

1. Go to https://share.streamlit.io
2. Sign in with your GitHub account (free)
3. Click "New app"
4. Select your repository: `YOUR_USERNAME/fac-synthesis-demo`
5. Set main file path: `app.py`
6. Click "Deploy"

Done! Your app will be live at: `https://YOUR-USERNAME-fac-synthesis-demo.streamlit.app`

### Why Streamlit Cloud?

- Completely free
- 5-minute setup
- Auto-deploy on git push
- Built-in HTTPS
- No server management

## Features

### SAE Configuration
- Load SAE weights from default path, Hugging Face, or upload files
- Adjust activation threshold (0.0 to 4.0)
- Support for LLaMA, Mistral, Qwen models

### Data Synthesizer
- Choose from LLaMA-3.1-8B, Mistral-7B, Qwen2-7B
- Optional GPT-4o-mini (no GPU required, API key needed)

### Core Functions
- Feature Analysis: Analyze text and visualize SAE activations
- Targeted Synthesis: Generate text that activates specific features
- FAC Coverage: Compute coverage metrics for datasets
- Batch Synthesis: Generate multiple samples efficiently

### Advanced
- Load custom datasets from Hugging Face
- Use your own SAE checkpoints
- Extend to new tasks

## Configuration

### GPU Requirements
- Local models (LLaMA/Mistral/Qwen): 16GB+ VRAM recommended
- GPT-4o-mini: No GPU needed (cloud API)

### SAE Weight Options

**Option 1: Default (Local)**
Place weights in:
```
demo/default_weights/Llama-3.1-8B-Instruct/sae_l16.pt
```

**Option 2: Hugging Face (Recommended)**
In the app:
1. Select "Hugging Face" as source
2. Enter repository ID: `username/sae-weights`
3. Enter filename: `sae_l16.pt`
4. Click "Load"

**Option 3: Upload**
Upload `.pt` or `.pth` files directly (max 200MB)

## Troubleshooting

**CUDA Out of Memory**
- Switch to GPT-4o-mini synthesizer
- Use smaller model (Mistral-7B)
- Close other GPU applications

**SAE Weights Not Found**
- Use Hugging Face option
- Check file path in `default_weights/`

**Import Errors**
```bash
pip install -r requirements.txt --upgrade
```

## Upload Your SAE to Hugging Face

```python
from huggingface_hub import HfApi

api = HfApi()
api.upload_file(
    path_or_fileobj="path/to/sae_l16.pt",
    path_in_repo="sae_l16.pt",
    repo_id="your-username/sae-weights",
    repo_type="model"
)
```

See `upload_to_hf_example.py` for more details.

## Citation

```bibtex
@article{less-is-enough-2026,
  title={Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs},
  author={...},
  year={2026}
}
```

## License

MIT License