AGENTS.md

AI Puppet Theater — Agent Notes

This repo is for AI Puppet Theater, a Hugging Face Gradio Space for the Build Small Hackathon.

The app should feel like a tiny improv stage: a user gives a premise, the app casts a few puppet characters, a Director manages a short scene, and the puppets perform with simple animation, audience interruptions, a transcript, and a behind-the-scenes trace.

These notes are here to help coding agents make useful changes without breaking the demo. The first priority is to get a working version live. Keep the code simple, keep the app reliable, and avoid adding advanced infrastructure before the core theatre loop works.

Product direction

AI Puppet Theater is a short, interactive, multi-agent puppet show.

A user enters a premise. The app creates a stage, casts puppet characters, and runs a short skit where the characters speak to each other. The user can interrupt the show by throwing props, summoning actors, heckling, or requesting a finale.

The experience should feel playful quickly. A user should be able to create a show and watch puppets perform without configuring models, signing in, or understanding the underlying architecture.

The stage should be the visual focus. Controls, transcripts, logs, traces, and model settings should support the experience without overwhelming it.

Skit pacing is part of the product. Shows should stay short by default. Prefer tight scenes, short lines, clear turn-taking, and a clean ending over long open-ended roleplay.

The product should grow in layers:

1. A reliable deterministic show engine.
2. LLM-generated actor lines.
3. A Director that manages pacing and speaker selection.
4. Actor agents with persona, goals, memory, and secrets.
5. Tool use for theatrical actions such as props, lights, sound effects, search, or memory lookup.
6. Trace logging, model experiments, fine-tuning, llama.cpp, and custom frontend polish.

Do not skip the reliable baseline. Advanced features should enhance the theater, not make the demo fragile.

Engineering priorities

Work in small steps. Each commit should leave the app runnable.

Prefer the smallest change that makes the current milestone work.

Use uv for local development. Keep requirements.txt generated for Hugging Face Spaces.

The deterministic backend should always work. Any model backend we add later should be optional and should fall back gracefully if loading or generation fails.

Prefer a simple Gradio Blocks app until a task explicitly asks for a custom frontend or gr.Server.

Do not introduce Docker, llama.cpp, model loading, fine-tuning, authentication, persistent storage, or heavy runtime changes unless the current task explicitly asks for them.

Suggested repo structure

app.py
pyproject.toml
uv.lock
requirements.txt
README.md
AGENTS.md

puppet_theater/
  __init__.py
  models.py
  session.py
  director.py
  actors.py
  stage.py
  trace.py
  backends.py
  prompts.py

assets/
  stage.css

This structure is a suggestion, not a strict rule. Keep the repo easy to understand.

Local commands

Install dependencies:

uv sync

Run the app locally:

uv run python app.py

For live reload while you edit app.py (including the CUSTOM_CSS string), use the Gradio CLI instead of python directly. It watches the project directory and restarts the server when the file changes:

uv run gradio app.py

The Blocks instance is named app; Gradio discovers it automatically. If you ever need to pin the name explicitly, use uv run gradio app.py --demo-name app.

Check syntax:

uv run python -m py_compile app.py puppet_theater/*.py

Generate Hugging Face Space requirements:

uv pip compile pyproject.toml -o requirements.txt

If tests are added, run:

uv run pytest

Coding style

Keep modules small and readable.

Use dataclasses or Pydantic models for core state. Avoid passing around loose dictionaries once the main app structure is in place.

Avoid secrets in code. Do not hard-code tokens, private URLs, or machine-specific paths.

Do not hide important behavior in global side effects. Lazy-loaded model backends are okay when needed, but the deterministic path should stay simple.

Do not introduce login or authentication unless explicitly asked.

Do not add heavy model dependencies unless the task explicitly asks for them.

Session model

The app should revolve around a theatre session with roughly these concepts:

• Show title
• User premise
• Setting
• Actors
• Current beat number
• Maximum beats
• Transcript
• Props
• Director log
• Trace events
• Finale requested flag

Actors should have:

• Name
• Avatar or emoji
• Goal
• Secret
• Speaking style
• Optional tools

Beats should have:

• Speaker
• Line
• Emotion
• Gesture
• Stage effect
• Optional tool request

Director behavior

The Director controls pacing. A simple six-beat structure is enough for the baseline:

1. Setup
2. Denial or contradiction
3. Evidence or prop
4. Secret reveal
5. Chaos or audience intervention
6. Finale

The Director should rotate speakers, avoid repetition, and end the scene cleanly.

Model and agent architecture

The deterministic backend is the baseline. It is a non-LLM fallback that uses simple rules or templates to create actors, advance beats, and generate short puppet lines. It keeps the app runnable when model backends fail or are unavailable.

Model backends should be optional and swappable. Possible backends include OpenBMB models, Ollama, llama.cpp, hosted inference, or a fine-tuned model. Do not add a backend unless the current task explicitly asks for it.

The app should degrade gracefully. If a model fails to load, times out, returns invalid JSON, or produces unusable text, fall back to the deterministic backend and record the issue in the logs.

Actor output should be short and stage-ready. When using an LLM, prefer structured output with fields such as line, emotion, gesture, stage_effect, and optional tool_request.

The Director is responsible for flow. It should choose the next speaker, decide the beat type, react to user interventions, avoid repetition, and end the scene cleanly.

Actors are responsible for character. Each actor should have a persona, goal, style, and optional secret. Later, actors may use tools, remember prior events, or react to stage state.

Tool use should serve the show. Tools should make the theater more fun or more agentic, not add complexity for its own sake.

When model output is used, prefer structured JSON and validate it. If the output is invalid, retry once if appropriate, then fall back to deterministic output.

A target actor output shape is:

{
  "line": "The moon denies everything, which is exactly what a guilty moon would do.",
  "emotion": "suspicious",
  "gesture": "point_accusingly",
  "stage_effect": "spotlight",
  "tool_request": null
}

Do not expose hidden reasoning. Logs and traces should show useful summaries of decisions, prompts, outputs, tool calls, errors, latency, and fallbacks, but they should not include secrets or private chain-of-thought.

UI guidance

Use Gradio Blocks as the default UI framework for now.

The page should roughly flow like this:

1. Title and short description
2. Premise input and create-show controls
3. Puppet stage
4. Audience controls
5. Transcript
6. Behind the Curtain log
7. Trace/debug information
8. Backend/model settings, if present

The active speaker should be obvious through animation, spotlight, glow, or speech bubble placement.

The stage should work on a normal laptop screen without requiring lots of scrolling before the user sees the action.

Browser testing

When changing layout or animation, run the app locally and inspect it in the Codex browser:

http://127.0.0.1:7860

Check that:

• The stage is visually dominant
• The active speaker is obvious
• Buttons are easy to find
• The transcript updates
• Audience actions visibly affect the show
• Reset returns the app to a clean state

Trace logging

Trace logging is useful for debugging now and for the hackathon “Sharing is Caring” badge later.

Log simple events such as:

• Show created
• Actor created
• Director decision
• Actor response
• Prop thrown
• Actor summoned
• Finale requested
• Fallback used

Do not log secrets, tokens, private file paths, or hidden reasoning.

Commit expectations

Each commit should represent a working milestone and leave the app runnable.

Use conventional, readable commit messages, for example:

• chore: add uv setup
• feat: add deterministic skit engine
• feat: render animated stage
• docs: update space readme

Before finishing a task, summarize what changed and which validation command passed.