---
title: Prompt Squirrel — Tag Retrieval System
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 5.43.1
python_version: 3.10.12
app_file: app.py
pinned: false
license: apache-2.0
short_description: Closed-vocabulary tag retrieval from natural language
emoji: 🧠
---

# Prompt Squirrel — Tag Retrieval System

Prompt Squirrel performs semantic search over a fixed image-tag vocabulary, mapping natural-language descriptions to relevant tags via retrieval and ranking.

The interface then allows users to interactively refine these retrieved tags into a final prompt.

### System Overview

This project demonstrates a multi-stage retrieval pipeline:
1. Local T5 query rewriting plus exact tag/alias phrase extraction
2. Structural inference and local classifier scoring
3. Candidate generation (TF-IDF + embeddings)
4. Candidate ranking and selection (closed-set LLM)
5. Interactive refinement (UI)

### Key Properties

- Closed vocabulary (no hallucinated tags)
- Retrieval-first architecture (no direct generation)
- Transparent intermediate stages for inspection and debugging
  
## What This Space Does

- Rewrites free text into retrieval-friendly query phrases with a local T5 model.
- Adds exact canonical tag and alias matches from the prompt to the retrieval query phrases.
- Uses structural inference and a local classifier to add high-confidence tags and extra reranker candidates.
- Retrieves candidate tags from a fixed vocabulary using semantic search.
- Ranks and filters candidates via closed-set LLM selection (no invented tags).
- Expands implied concepts and organizes results into editable groups.
- Lets users refine and assemble the final prompt via interactive controls.

## How To Use

1. Enter an image description in `Enter Prompt`.
2. Click `Run`.
3. Toggle tags in rows to add/remove them.
4. Copy the final text from `Suggested Prompt`.

## Technologies Used

- FastText embeddings for semantic retrieval.
- HNSW indexes for efficient nearest-neighbor search.
- Reduced TF-IDF vectors for context-aware ranking.
- Local T5 model for query reformulation.
- Local ModernBERT multi-label classifier for calibrated high-confidence tags and reranker candidates.
- OpenRouter instruction models for structural inference and closed-set selection.
  Default model: `mistralai/mistral-small-24b-instruct-2501` (chosen empirically on internal caption-evident evaluations; configurable).
- Gradio for the interactive web UI.

## Current Snapshot

Latest relabel-aware rescore on caption-evident n=30:

- Micro precision/recall/F1: `0.554 / 0.750 / 0.638`
- Macro precision/recall/F1: `0.564 / 0.764 / 0.633`
- Ground-truth scale: 30 images, 440 total tag assignments, 205 unique tags

Latency from local UI timing logs (n=110 runs):

- Median: `8.56s`
- P75: `12.56s`
- P90: `18.33s`

## Documentation And Contracts

- Full architecture and rationale (same source used by the in-app docs accordion): [docs/space_overview.md](docs/space_overview.md)
- Retrieval contract: [docs/retrieval_contract.md](docs/retrieval_contract.md)
- Rewrite contract: [docs/rewrite_contract.md](docs/rewrite_contract.md)
- Closed-set selection contract: [docs/stage3_contract.md](docs/stage3_contract.md)