| --- |
| title: ShotCraft |
| emoji: 🎬 |
| colorFrom: purple |
| colorTo: pink |
| sdk: gradio |
| app_file: app.py |
| license: apache-2.0 |
| short_description: AI art director for e-commerce product shots |
| tags: |
| - backyard-ai |
| - modal |
| --- |
| |
| # 🎬 ShotCraft — AI Shot Director |
|
|
| Upload a product photo → an 8B vision model designs 5 marketing shot |
| concepts grounded in your *actual* product → a 12B image model renders |
| them as production-ready stills. |
|
|
| **Build Small Hackathon 2026** | Models: [MiniCPM-V-2_6](https://huggingface.co/openbmb/MiniCPM-V-2_6) (8B) + [FLUX.1-schnell](https://huggingface.co/black-forest-labs/FLUX.1-schnell) (12B) — both ≤32B ✅ |
|
|
| ## Pipeline |
| 1. Upload product photo + brand description + category + style preset |
| 2. MiniCPM-V-2_6 analyzes the photo → 5 shot concepts (scene, camera, lighting, palette, props, marketing angle, optimized prompt) |
| 3. Review & edit concepts |
| 4. FLUX.1-schnell renders 5 stills → pick hero frames → export ZIP |
| |
| Team: Pawel (Stage 1) · Rafal (Stage 2) · Stefan (integration) |
| |
| ## Architecture — powered by Modal ⚡ |
| |
| This Space is the Gradio interface; **all model inference runs on |
| [Modal](https://modal.com)** (`modal_backend/shotcraft_inference.py` in the |
| [repo](https://github.com/rafalbog/nice-day-pink-panther)): |
| |
| | Stage | Model | Modal GPU | |
| |---|---|---| |
| | 1 — Shot Director | MiniCPM-V-2_6 (8B, bf16) | A10G | |
| | 2 — Frame Generator | FLUX.1-schnell (12B, bf16, 4-step) | L40S | |
|
|
| - One HTTP call per stage; the whole 5-frame reel renders in a single |
| batched call, per-frame regen is a 1-frame call (seeded, FR-2.3/2.4). |
| - Weights cached on a Modal Volume; containers stay warm 5 min between calls. |
| - The Space needs `SHOTCRAFT_API_URL` → the Modal endpoint. The deploy |
| workflow can set it automatically from `SHOTCRAFT_MODAL_WORKSPACE`, |
| `SHOTCRAFT_MODAL_APP`, and `SHOTCRAFT_MODAL_FUNCTION`. |
|
|
| ## Dev notes |
| - **Backend deploy:** `modal deploy modal_backend/shotcraft_inference.py` |
| (or `modal serve` for a dev loop). Health check: `GET /health`. |
| - **Tests:** `python test_smoke.py` (pure logic, no backend) and |
| `python test_app_e2e.py` (real end-to-end against the deployed backend — |
| spends GPU time). |
| - **Plan:** see `docs/shotcraft-IMPLEMENTATION_PLAN.md` for the slice-by-slice build order. |
|
|
| ## Deployment configuration |
|
|
| Keep personal tokens in GitHub Secrets, not in code. This shared repo deploys |
| one shared Hugging Face Space, but can point that Space at Rafal's or Pawel's |
| Modal backend. Manual workflow runs ask for `deployment_profile` (`rafal` or |
| `pawel`). Pushes to `main` still deploy the default `rafal` profile. |
|
|
| Rafal uses the existing GitHub Secrets: |
| - `HF_TOKEN` — Hugging Face write token for the target Space. |
| - `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` — Modal token for the target workspace. |
|
|
| Pawel uses prefixed GitHub Secrets for Modal only: |
| - `PAWEL_MODAL_TOKEN_ID` |
| - `PAWEL_MODAL_TOKEN_SECRET` |
|
|
| Rafal/default GitHub Variables: |
| - `HF_USERNAME` — username used for HF git auth. Defaults to `rafalbog`. |
| - `HF_SPACE_REPO` — Space repo id, for example `build-small-hackathon/ai-video-generation`. |
| - `SHOTCRAFT_MODAL_WORKSPACE` — Modal workspace slug. Defaults to `rafalbogusdxc`. |
| - `SHOTCRAFT_MODAL_APP` — Modal app name. Defaults to `shotcraft-inference`. |
| - `SHOTCRAFT_MODAL_FUNCTION` — Modal ASGI function name. Defaults to `api`. |
| - `SHOTCRAFT_API_URL` — full backend URL override. If unset, the workflow builds |
| `https://<workspace>--<app>-<function>.modal.run`. |
| - `SHOTCRAFT_HF_SECRET_NAME` — Modal secret that contains the HF token for model |
| downloads. Defaults to `huggingface-secret`. |
| - `SHOTCRAFT_HF_CACHE_VOLUME` — Modal volume for cached weights. Defaults to |
| `shotcraft-hf-cache`. |
|
|
| Pawel-specific GitHub Variables use the same names with a `PAWEL_` prefix: |
| - `PAWEL_SHOTCRAFT_MODAL_WORKSPACE` |
| - optional: `PAWEL_SHOTCRAFT_API_URL` |
| - optional: `PAWEL_SHOTCRAFT_MODAL_APP` |
| - optional: `PAWEL_SHOTCRAFT_MODAL_FUNCTION` |
| - optional: `PAWEL_SHOTCRAFT_HF_SECRET_NAME` |
| - optional: `PAWEL_SHOTCRAFT_HF_CACHE_VOLUME` |
|
|
| For Rafal, the existing repository-level secrets/variables can stay as-is. For |
| Pawel, add the prefixed Modal secrets/variables under the same repository |
| Actions settings page, then run either deploy workflow manually and choose |
| `deployment_profile=pawel`. The frontend workflow still uses the existing |
| `HF_TOKEN` to update the shared Space at |
| `build-small-hackathon/ai-video-generation`. |
|
|
