Deployment Setup

multimodal-engine/.env.example

@@ -0,0 +1,7 @@
+# Google Gemini API key — required for transcription, scene analysis, blog synthesis, and clip verification
+# Get yours at: https://aistudio.google.com/app/apikey
+GEMINI_API_KEY="your_google_gemini_api_key_here"
+
+# OpenAI API key — used as a fallback if Gemini scene analysis fails (optional but recommended)
+# Get yours at: https://platform.openai.com/api-keys
+OPENAI_API_KEY="your_openai_api_key_here"

multimodal-engine/.gitignore

@@ -5,4 +5,6 @@ data/*.avi
 data/extracted_frames
 data/extracted_clips
 data/extracted_real
-output/clips/*.mp4
+output/clips/*.mp4
+assets/video.mp4
+multimodal-engine/assets/video.mp4

multimodal-engine/Dockerfile

@@ -0,0 +1,28 @@
+FROM python:3.12-slim
+
+# Install system utilities and FFmpeg video processing binaries
+RUN apt-get update && apt-get install -y \
+    ffmpeg \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+# Set up a non-root user to satisfy Hugging Face security protocols
+RUN useradd -m -u 1000 user
+USER user
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH
+
+WORKDIR $HOME/app
+
+# Copy local dependency maps into virtual container layers
+COPY --chown=user requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy the rest of the application codebase
+COPY --chown=user . .
+
+# Hugging Face Spaces strictly requires port 7860
+EXPOSE 7860
+
+# Run Streamlit on Hugging Face's required port configuration layout
+ENTRYPOINT ["streamlit", "run", "multimodal-engine/app/app.py", "--server.port=7860", "--server.address=0.0.0.0"]

multimodal-engine/README.md

@@ -1,186 +1,420 @@
-# 🎬 SynapseMedia: Multimodal Content Engine
+<div align="center">
 
-An AI-powered pipeline that takes a raw video file and automatically produces a full content package — audio transcript, visual scene breakdown, a generated technical blog post, a 16:9 highlight clip, and a 9:16 vertical short reel.
+<img src="assets/dashboard.png" alt="SynapseMedia Dashboard" width="100%"/>
+
+# 🎬 SynapseMedia — Multimodal Content Engine
+
+**Turn any long-form video into a vertical reel and a CMS-ready blog post — fully automated.**
+
+[![Python](https://img.shields.io/badge/Python-3.12-blue?logo=python&logoColor=white)](https://www.python.org/)
+[![Streamlit](https://img.shields.io/badge/Streamlit-1.57-FF4B4B?logo=streamlit&logoColor=white)](https://streamlit.io/)
+[![Gemini](https://img.shields.io/badge/Gemini_2.5-Flash_%26_Pro-4285F4?logo=google&logoColor=white)](https://ai.google.dev/)
+[![Tests](https://img.shields.io/badge/Tests-72_passing-brightgreen?logo=pytest)](multimodal-engine/test/)
+[![Docker](https://img.shields.io/badge/Docker-Ready-2496ED?logo=docker&logoColor=white)](Dockerfile)
+[![License](https://img.shields.io/badge/License-MIT-yellow)](LICENSE)
+
+[**Live Demo**](#-demo) · [**Quick Start**](#-quick-start) · [**Architecture**](#-architecture) · [**Docker**](#-docker-deployment)
+
+</div>
+
+---
+
+## ✨ What It Does
+
+Upload an MP4 — a tutorial, podcast, or product walkthrough. SynapseMedia runs it through a 4-phase AI pipeline and delivers two production-ready assets:
+
+| Output | Description |
+|---|---|
+| 📝 **Blog Post** | Full Markdown article with YAML front matter, headers, and tables — ready for WordPress, Dev.to, or Hugo |
+| 📱 **Vertical Reels** | 1080×1920 MP4 clips at 9:16 — verified for energy score, auto-cropped, ready for TikTok, Reels, or Shorts |
+
+Everything runs with **zero manual editing**. No timeline scrubbing, no copy-pasting transcripts, no cropping by hand.
 
 ---
 
-## 🧠 What It Does
+## 🎥 Demo
+
+<div align="center">
 
-Upload any tutorial or walkthrough video. The engine runs three sequential AI phases:
+https://github.com/ajme-abes/RAG-Multimodal-SafeAI/tree/main/multimodal-engine/assets/video.mp4
 
-- **Phase 1 — Audio:** Demuxes the audio stream via FFmpeg and transcribes it using Gemini 2.5 Flash.
-- **Phase 2 — Vision:** Extracts keyframes at a configurable interval and runs chronological scene analysis via Gemini Vision (with GPT-4o-mini as fallback).
-- **Phase 3 — Synthesis:** Fuses the transcript and visual breakdown into a structured Markdown blog post, then cuts a horizontal highlight clip and a vertical mobile short reel.
+</div>
+
+> Upload a video → click one button → get a blog post and vertical reels in minutes.
 
 ---
 
-## 📂 Project Structure
+## 📸 Screenshots
+
+<table>
+  <tr>
+    <td align="center" width="33%">
+      <img src="assets/dashboard.png" alt="Main Dashboard" width="100%"/>
+      <br/><b>Main Dashboard</b>
+      <br/><sub>Upload panel, interval slider, layout mode selector</sub>
+    </td>
+    <td align="center" width="33%">
+      <img src="assets/mdpage.png" alt="Blog Post Output" width="100%"/>
+      <br/><b>Generated Blog Post</b>
+      <br/><sub>YAML front matter + full Markdown rendered in-app</sub>
+    </td>
+    <td align="center" width="33%">
+      <img src="assets/reelpage.png" alt="Reels Output" width="100%"/>
+      <br/><b>Vertical Reels</b>
+      <br/><sub>9:16 clips displayed side-by-side with inline playback</sub>
+    </td>
+  </tr>
+</table>
+
+---
+
+## 🏗️ Architecture
+
+The pipeline runs two tracks in parallel — one for audio, one for video — then fuses them.
 
 ```
-multimodal-engine/
-├── app/
-│   ├── app.py               # Streamlit UI — main entry point
-│   ├── workflow_engine.py   # Orchestrates the full pipeline & blog synthesis
-│   ├── audio_processor.py   # FFmpeg audio extraction + Gemini transcription
-│   ├── video_processor.py   # FFmpeg keyframe extraction + Gemini/OpenAI scene analysis
-│   ├── clip_extractor.py    # Cuts a 16:9 horizontal highlight clip
-│   └── reel_generator.py    # Crops and compiles a 9:16 vertical short reel
-├── data/
-│   ├── sample.mp4                        # Test video input
-│   ├── extracted_audio.mp3               # Audio demux output
-│   ├── extracted_frames/                 # Keyframe JPGs
-│   ├── extracted_clips/                  # Output clips
-│   └── extracted_real/                   # Output vertical reels
-├── output/                               # Generated blog posts (.md)
-├── requirements.txt
-└── README.md
+┌─────────────────────────────────────┐
+│           Uploaded MP4 Video        │
+└──────────────┬──────────────────────┘
+               │
+     ┌─────────┴──────────┐
+     ▼                    ▼
+ TRACK A               TRACK B
+ audio_processor       video_processor
+ ─────────────         ───────────────
+ FFmpeg demux          FFmpeg keyframe
+ 16kHz PCM WAV         extraction (JPEGs)
+     │                      │
+     ▼                      ▼
+ Gemini 2.5 Flash      Gemini 2.5 Flash
+ StructuredTranscript  ChronologicalVisual
+ (float timestamps)    Timeline (float ts)
+     │                      │
+     └──────────┬───────────┘
+                │
+                ▼
+        workflow_engine.py
+        ───────────────────
+        Injects both tracks
+        into Gemini prompt
+        → Markdown Blog Post
+        (YAML front matter)
+                │
+                ▼
+        agent_optimizer.py
+        ──────────────────
+        Stage 1: Text filter
+        (Gemini 2.5 Flash)
+        Top clip candidates
+                │
+                ▼
+        reel_generator.py
+        ─────────────────
+        Stage 2: Video verify
+        (Gemini 2.5 Pro)
+        Energy score > 60
+        Timestamp fine-tune
+                │
+                ▼
+        video_processor.py
+        ──────────────────
+        FFmpeg 9:16 render
+        Blurred Stack or
+        AI Smart Face Crop
+                │
+       ┌────────┴────────┐
+       ▼                 ▼
+  📝 Blog Post      📱 Vertical Reels
+  (.md + YAML)      (1080×1920 MP4)
 ```
 
+### The 4 Phases
+
+| Phase | Module | What Happens |
+|---|---|---|
+| **1 — Hear** | `audio_processor.py` | FFmpeg extracts 16kHz mono WAV → Gemini 2.5 Flash transcribes into `StructuredTranscript` with per-word float timestamps |
+| **2 — See** | `video_processor.py` | FFmpeg samples keyframes at configurable intervals → Gemini 2.5 Flash maps each frame to a float timestamp in `ChronologicalVisualTimeline` |
+| **3 — Write** | `workflow_engine.py` | Both data streams are serialised and injected into a Gemini prompt → structured Markdown blog post with YAML front matter |
+| **4 — Cut** | `agent_optimizer.py` → `reel_generator.py` | Stage 1 text filter finds hook candidates → Stage 2 uploads raw clips to Gemini 2.5 Pro for energy scoring → FFmpeg renders verified clips |
+
 ---
 
-## ⚙️ Setup
+## ⚡ Key Features
+
+**🎯 Zero-Hallucination Timestamps**
+Pydantic models enforce `float` types at every pipeline boundary. No `HH:MM:SS` strings, no off-by-one second errors, no silent type mismatches.
+
+**🔄 Dual Render Modes**
+- **Blurred Stack** — Scales the full 16:9 frame to fit 9:16, fills margins with a blurred duplicate. Keeps code and UI text fully readable.
+- **AI Smart Face Crop** — Gemini detects speaker position (`left` / `center` / `right`) and centers the crop frame around them.
+
+**🛡️ Two-Stage Clip Verification**
+Every candidate clip is scored by Gemini 2.5 Pro before rendering. Only clips with `audio_energy_score > 60` and `final_relevance_decision = true` make it to FFmpeg.
+
+**♻️ Automatic Fallback Chain**
+If Gemini scene analysis fails → OpenAI GPT-4o-mini takes over → if that fails too → a placeholder timeline keeps the pipeline alive.
+
+**⏳ Exponential Backoff with Jitter**
+Every Gemini call is wrapped in `retry_with_backoff` — handles 429 / 503 errors automatically with randomised delay to prevent thundering herd.
+
+**☁️ Cloud Asset Cleanup**
+All files uploaded to the Gemini Files API are deleted inside `finally` blocks — no leaked cloud storage, even on exception.
+
+**📝 CMS-Ready Output**
+Blog posts include complete YAML front matter: `title`, `slug`, `date`, `tags`, `category`, `description`. Date-prefixed filenames prevent overwrites.
+
+---
+
+## 🛠️ Tech Stack
+
+| Layer | Tool | Version |
+|---|---|---|
+| **UI** | Streamlit | 1.57.0 |
+| **Audio extraction** | FFmpeg (`pcm_s16le`) | system |
+| **Transcription** | Google Gemini 2.5 Flash | via `google-genai` 1.68.0 |
+| **Scene analysis** | Google Gemini 2.5 Flash | via `google-genai` 1.68.0 |
+| **Vision fallback** | OpenAI GPT-4o-mini | via `openai` 2.37.0 |
+| **Clip verification** | Google Gemini 2.5 Pro | via `google-genai` 1.68.0 |
+| **Blog synthesis** | Google Gemini 2.5 Flash | via `google-genai` 1.68.0 |
+| **Video rendering** | FFmpeg (`libx264` + `aac`) | system |
+| **Schema validation** | Pydantic v2 | 2.12.5 |
+| **Testing** | pytest | 72 tests, 0 failures |
+
+---
+
+## 🚀 Quick Start
 
 ### 1. Prerequisites
 
-- Python 3.10+
-- **FFmpeg** must be installed and available in your system PATH
-  - Windows: [ffmpeg.org/download](https://ffmpeg.org/download.html) or `winget install ffmpeg`
-  - macOS: `brew install ffmpeg`
-  - Linux: `sudo apt install ffmpeg`
+Install **FFmpeg** and make sure it's on your system PATH:
+
+```bash
+# macOS
+brew install ffmpeg
+
+# Ubuntu / Debian
+sudo apt install ffmpeg
 
-### 2. Install dependencies
+# Windows
+winget install ffmpeg
+
+# Verify
+ffmpeg -version
+```
+
+### 2. Clone & Install
 
 ```bash
-cd multimodal-engine
+git clone https://github.com/ajme-abes/RAG-Multimodal-SafeAI
+cd synapsemedia/multimodal-engine
+
+# Create a virtual environment
+python -m venv venv
+source venv/bin/activate      # Windows: venv\Scripts\activate
+
+# Install pinned dependencies
 pip install -r requirements.txt
 ```
 
-### 3. Environment variables
+### 3. Configure API Keys
 
-Create a `.env` file in the root of the repository (or in `multimodal-engine/`):
+Copy the example env file and fill in your keys:
 
-```env
-GOOGLE_API_KEY=your_google_gemini_key
-OPENAI_API_KEY=your_openai_key
+```bash
+cp .env.example .env
 ```
 
-Both keys are used — Gemini is the primary model, OpenAI GPT-4o-mini is the visual analysis fallback.
+Edit `.env`:
 
----
+```env
+GEMINI_API_KEY="your_google_gemini_api_key"
+OPENAI_API_KEY="your_openai_api_key"
+```
 
-## 🚀 Running the App
+> Get your Gemini key at [aistudio.google.com](https://aistudio.google.com/app/apikey)
+> The OpenAI key is optional — used only as a fallback if Gemini scene analysis fails.
+
+### 4. Run
 
 ```bash
-cd multimodal-engine/app
-streamlit run app.py
+streamlit run app/app.py
 ```
 
-Then open [http://localhost:8501](http://localhost:8501) in your browser.
+Open **http://localhost:8501** in your browser.
 
-### UI Walkthrough
+---
 
-1. Drag and drop an `.mp4` file into the sidebar uploader
-2. Set the **keyframe sampling interval** (seconds between captured frames)
-3. Click **🚀 Process Complete AI Workflow**
-4. View results across three tabs:
-   - **📄 Generated Blog Post** — download the Markdown article
-   - **🎬 Automated Video Assets** — preview the 16:9 clip and 9:16 reel side by side
-   - **🎙️ Raw Data Tracks** — inspect the raw transcript and scene analysis text
+## 🎛️ Usage Guide
+
+<table>
+  <tr>
+    <th>Step</th>
+    <th>Action</th>
+    <th>Tip</th>
+  </tr>
+  <tr>
+    <td>1</td>
+    <td>Upload an MP4 in the left sidebar</td>
+    <td>Works best with videos 3–60 minutes long</td>
+  </tr>
+  <tr>
+    <td>2</td>
+    <td>Set the <b>Downsampling Interval</b> slider</td>
+    <td>Use <b>5s</b> for code tutorials, <b>10–15s</b> for interviews</td>
+  </tr>
+  <tr>
+    <td>3</td>
+    <td>Choose a <b>Reel Layout</b></td>
+    <td><b>Blurred Stack</b> for screen recordings, <b>AI Smart Crop</b> for talking-head</td>
+  </tr>
+  <tr>
+    <td>4</td>
+    <td>Click <b>🚀 Process Complete AI Workflow</b></td>
+    <td>A 10-min video takes roughly 2–4 minutes to process</td>
+  </tr>
+  <tr>
+    <td>5</td>
+    <td>Download from the output tabs</td>
+    <td><b>Blog Post</b> tab → Markdown file · <b>Reels</b> tab → MP4 files</td>
+  </tr>
+</table>
+
+### Output Tabs
+
+| Tab | Content |
+|---|---|
+| 📄 **Blog Post** | Rendered Markdown with download button |
+| 📱 **Mobile Reels** | Inline video player for each 9:16 clip |
+| 🎙️ **System Logs** | Raw JSON for the transcript and visual timeline |
 
 ---
 
-## 🔧 Running Modules Standalone
+## 🐳 Docker Deployment
 
-Each module has its own `__main__` block for isolated testing:
+### Build & Run Locally
 
 ```bash
-# Test audio extraction + transcription only
-python app/audio_processor.py
+docker build -t synapsemedia .
+docker run -p 8501:8501 \
+  -e GEMINI_API_KEY="your_key" \
+  -e OPENAI_API_KEY="your_key" \
+  synapsemedia
+```
 
-# Test keyframe extraction + scene analysis only
-python app/video_processor.py
+Open **http://localhost:8501**.
+
+### Docker Compose (Recommended)
+
+```yaml
+version: "3.9"
+services:
+  synapsemedia:
+    build: .
+    ports:
+      - "8501:8501"
+    environment:
+      - GEMINI_API_KEY=${GEMINI_API_KEY}
+      - OPENAI_API_KEY=${OPENAI_API_KEY}
+    volumes:
+      - ./data:/workspace/data
+      - ./output:/workspace/output
+```
 
-# Test clip cutting only
-python app/clip_extractor.py
+```bash
+docker compose up --build
+```
 
-# Test vertical reel generation only
-python app/reel_generator.py
+---
 
-# Run the full pipeline headlessly (no UI)
-python app/workflow_engine.py
+## 🧪 Tests
+
+The test suite runs without any API keys or video files.
+
+```bash
+# From the multimodal-engine/ directory
+pytest
 ```
 
-All standalone runners expect `../data/sample.mp4` as the input file.
+**72 tests · 0 failures · ~30s runtime**
+
+| File | What It Covers |
+|---|---|
+| `test_models.py` | Pydantic schema construction, type coercion, JSON roundtrip |
+| `test_utils.py` | Retry on 429/503, immediate raise on 400/401, backoff delay doubling |
+| `test_video_processor.py` | All 4 FFmpeg render modes, directory cleanup, base64 encoding, input validation |
+| `test_agent_optimizer.py` | Slug sanitisation, layout routing, per-clip render calls, error isolation |
+| `test_pipeline_integration.py` | All 4 phases mocked end-to-end, fallback chains, full pipeline smoke test |
 
 ---
 
-## 🏗️ Architecture
+## 📁 Project Structure
 
 ```
-Video Input (.mp4)
-      │
-      ├──[FFmpeg]──► extracted_audio.mp3
-      │                    │
-      │              [Gemini 2.5 Flash]
-      │                    │
-      │              Audio Transcript
-      │
-      ├──[FFmpeg]──► keyframe_XXXX.jpg (every N seconds)
-      │                    │
-      │         [Gemini Vision / GPT-4o-mini fallback]
-      │                    │
-      │              Visual Scene Breakdown
-      │
-      └──[Gemini 2.5 Flash]──► Technical Blog Post (.md)
-      │
-      ├──[FFmpeg]──► 16:9 Highlight Clip (.mp4)
-      └──[FFmpeg]──► 9:16 Vertical Short Reel (.mp4)
+multimodal-engine/
+├── app/
+│   ├── app.py                    # Streamlit dashboard — pipeline orchestration & UI
+│   ├── models.py                 # Shared Pydantic schemas (single source of truth)
+│   ├── utils.py                  # Exponential backoff with jitter
+│   ├── audio_processor.py        # Phase 1 — FFmpeg WAV demux + Gemini transcription
+│   ├── video_processor.py        # Phase 2 — Keyframe extraction + VLM scene analysis + FFmpeg render
+│   ├── clip_extractor.py         # Phase 4a — Stage 1 text-based semantic filter
+│   ├── reel_generator.py         # Phase 4b — Stage 2 multi-modal video verification
+│   └── workflow_engine.py        # Phase 3 — Blog synthesis + pipeline orchestration
+├── assets/
+│   ├── dashboard.png             # UI screenshot — main dashboard
+│   ├── mdpage.png                # UI screenshot — blog output tab
+│   ├── reelpage.png              # UI screenshot — reels output tab
+│   └── video.mp4                 # Demo walkthrough video
+├── data/                         # Runtime: uploaded videos, audio, keyframes (gitignored)
+├── output/                       # Runtime: blog posts and rendered reels (gitignored)
+├── test/
+│   ├── __init__.py
+│   ├── test_models.py
+│   ├── test_utils.py
+│   ├── test_video_processor.py
+│   ├── test_agent_optimizer.py
+│   └── test_pipeline_integration.py
+├── .env.example                  # API key template
+├── Dockerfile                    # Production container
+├── pytest.ini                    # Test runner config
+└── requirements.txt              # Pinned Python dependencies
 ```
 
 ---
 
-## 🛠️ Tech Stack
+## 📈 Roadmap
 
-| Component         | Technology                          |
-|-------------------|-------------------------------------|
-| UI                | Streamlit                           |
-| Audio extraction  | FFmpeg (`libmp3lame`)               |
-| Transcription     | Google Gemini 2.5 Flash             |
-| Keyframe slicing  | FFmpeg (`fps` filter)               |
-| Scene analysis    | Google Gemini 2.5 Flash Vision      |
-| Vision fallback   | OpenAI GPT-4o-mini                  |
-| Blog synthesis    | Google Gemini 2.5 Flash             |
-| Clip/reel cutting | FFmpeg (`libx264`, crop filter)     |
-| Environment       | python-dotenv                       |
+- [ ] **Word-level animated subtitles** — Burn styled captions into frames using ASS/SSA overlay filters
+- [ ] **Frame-by-frame face tracking** — Replace static `left/center/right` with a landmark model for smooth pan-and-scan
+- [ ] **Social publishing webhooks** — Direct push to YouTube Shorts, Instagram Reels, and TikTok APIs
+- [ ] **Natural language video search** — Query timestamps using the `StructuredTranscript` as a semantic index
+- [ ] **Configurable resolutions** — 1080×1920 (TikTok/Reels), 1080×1080 (square), custom aspect ratios
+- [ ] **Progress persistence** — Save pipeline state to disk so a page refresh doesn't lose results
 
 ---
 
-## 📈 Roadmap
+## ⚠️ Known Limitations
 
-- [x] FFmpeg audio demux
-- [x] Gemini audio transcription
-- [x] FFmpeg keyframe extraction
-- [x] Gemini visual scene analysis
-- [x] OpenAI vision fallback
-- [x] Blog post synthesis (transcript + visuals)
-- [x] 16:9 highlight clip cutter
-- [x] 9:16 vertical reel generator
-- [x] Streamlit UI with 3-phase progress tracking
-- [ ] Smart clip detection (auto-find highlight moments from transcript)
-- [ ] Subtitle/caption burn-in on reels
-- [ ] Cross-modal search (natural language → video timestamp)
-- [ ] Batch processing (queue multiple videos)
-- [ ] FastAPI backend
+- **Gemini file processing delay** — Uploaded video files enter a `PROCESSING` state before they can be queried. Large clips can take 30–60 seconds to process server-side.
+- **No persistent storage** — Streamlit session state resets on page refresh. Results are saved to `output/` on disk but not re-loaded automatically.
+- **Rate limits** — Gemini 2.5 Pro (used in Stage 2) has lower rate limits than Flash. If you process many clips back-to-back, the backoff utility will kick in.
 
 ---
 
-## 🖼️ Screenshots
+## 🤝 Contributing
 
-> Screenshots will be added once the UI is deployed.
+1. Fork the repo
+2. Create a feature branch: `git checkout -b feature/my-feature`
+3. Make your changes and add tests
+4. Run `pytest` — all tests must pass
+5. Open a pull request
 
 ---
 
-## 👤 Author
+<div align="center">
+
+Built with ❤️ using **Gemini 2.5**, **FFmpeg**, and **Streamlit**
+
+⭐ Star this repo if it saved you hours of manual editing
 
-Built as Phase 2 of a structured AI engineering roadmap.  
-See the [root README](../README.md) for the full portfolio overview.
+</div>

multimodal-engine/app/agent_optimizer.py

@@ -10,6 +10,18 @@ from audio_processor import StructuredTranscript
 from video_processor import ChronologicalVisualTimeline, generate_vertical_reel_clip
 from utils import retry_with_backoff
 
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
+
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
+
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
 load_dotenv()
 
 # =====================================================================
@@ -74,18 +86,17 @@ def discover_highlights_autonomously(
     try:
         print("🤖 Requesting Gemini processing with zero-hallucination Pydantic constraint filters...")
 
-        def excute_call():
+        def execute_call():
             return client.models.generate_content(
                 model="gemini-2.5-flash",
-            contents=[macro_prompt],
-            config=types.GenerateContentConfig(
-                response_mime_type="application/json",
-                response_schema=HighlightAnalysisResult,
-                temperature=0.1, # Keep temperature low to prevent numeric drift
-            )
-
+                contents=[macro_prompt],
+                config=types.GenerateContentConfig(
+                    response_mime_type="application/json",
+                    response_schema=HighlightAnalysisResult,
+                    temperature=0.1,
+                ),
             )
-        response = retry_with_backoff(excute_call)
+        response = retry_with_backoff(execute_call)
 
         # Access the typed object directly using response.parsed to avoid raw json parsing errors
         validated_data: HighlightAnalysisResult = response.parsed
@@ -114,12 +125,11 @@ def run_autonomous_editing_pipeline(
     if not discovered_highlights:
         return False
         
-    os.makedirs("../output/clips", exist_ok=True)
     
     for index, highlight in enumerate(discovered_highlights):
         print(f"🎬 Processing Clip #{index + 1}: [{highlight.hook_title}]")
         safe_title = "".join(c for c in highlight.hook_title if c.isalnum() or c in (' ', '_')).rstrip().replace(' ', '_').lower()
-        output_reel = f"../output/clips/auto_reel_{index + 1}_{safe_title}.mp4"
+        output_reel = os.path.join(CLIPS_DIR, f'auto_reel_{index + 1}_{safe_title}.mp4')
         
         # Determine the string instruction parameter for FFmpeg
         # If user picked Blurred Stack, override position and pass "blurred"

multimodal-engine/app/app.py

@@ -1,30 +1,37 @@
-import os
+import os, re
 import sys
 import glob
+import time
 import streamlit as st
-
-# Setup system environment alignment routes
-sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
-
+from datetime import datetime
 from workflow_engine import generate_production_blog
 from audio_processor import extract_audio_from_video, transcribe_audio
 from video_processor import extract_keyframes, analyze_scene_with_gemini
 from agent_optimizer import run_autonomous_editing_pipeline
-
 from models import StructuredTranscript, ChronologicalVisualTimeline
 
-# Configure clean, production-grade page layout parameters
-st.set_page_config(page_title="Multimodal Content Engine", page_icon="🎬", layout="wide")
+# Setup system environment alignment routes
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
 
-# Sync runtime directory configurations with core backends
-DATA_DIR = "../data"
-FRAMES_DIR = "../data/extracted_frames"
-OUTPUT_CLIPS_DIR = "../output/clips"
-BLOG_OUTPUT_PATH = "../output/how_multimodals_work_blog.md"
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+FRAMES_DIR = os.path.join(DATA_DIR, "extracted_frames")
+OUTPUT_CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
 
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
 os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
 os.makedirs(FRAMES_DIR, exist_ok=True)
 os.makedirs(OUTPUT_CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
+
+# Configure clean, production-grade page layout parameters
+st.set_page_config(page_title="Multimodal Content Engine", page_icon="🎬", layout="wide")
+
 
 st.title("🎬 Multimodal AI Content Engine")
 st.caption("Convert long horizontal streams into clear technical blog posts and vertical mobile shorts using a type-safe pipeline.")
@@ -50,7 +57,6 @@ with st.sidebar:
         help="Capture 1 image frame every X seconds of timeline playback."
     )
     
-    # 🆕 UPGRADE: User choices for dynamic reel layout style
     reel_style = st.selectbox(
         "📱 Reel Visual Layout Style",
         options=["Blurred Stack Mode (Presentation/Code)", "AI Smart Face Crop (Podcast/Vlog)"],
@@ -89,27 +95,41 @@ if uploaded_video is not None:
             extract_audio_from_video(video_input_path, audio_output_path)
             st.write("Transcribing audio tracks into structured Pydantic time models...")
             st.session_state.transcript_obj = transcribe_audio(audio_output_path)
+            if st.session_state.transcript_obj is None:
+                status.update(label="Phase 1 Failed: Transcription returned no data.", state="error")
+                st.error("❌ Audio transcription failed — Gemini returned no structured output. Check your GEMINI_API_KEY and try again.")
+                st.stop()
             status.update(label="Phase 1 Complete: Audio Transcript Secured!", state="complete")
-            
+
         # Phase 2 Step: Seeing
         with st.status("🎬 Phase 2: Extracting Timelines & Scene Layouts...", expanded=True) as status:
             st.write("Slicing uniform visual image arrays from video timeline...")
             extract_keyframes(video_input_path, FRAMES_DIR, interval_seconds=sampling_interval)
             st.write("Analyzing chronological frame sequences via Vision-VLM array...")
             st.session_state.visual_breakdown_obj = analyze_scene_with_gemini(FRAMES_DIR, interval_seconds=sampling_interval)
+            if st.session_state.visual_breakdown_obj is None:
+                status.update(label="Phase 2 Failed: Scene analysis returned no data.", state="error")
+                st.error("❌ Visual timeline extraction failed — no frames were found or both Gemini and OpenAI fallback failed.")
+                st.stop()
             status.update(label="Phase 2 Complete: Visual Timeline Extracted!", state="complete")
-            
+
         # Phase 3 Step: Fusing and Clipping
         with st.status("🧠 Phase 3: Synthesizing Content & Cutting Highlights...", expanded=True) as status:
             st.write("Fusing text arrays and structural visual timelines together into markdown entries...")
-            st.session_state.final_blog = generate_production_blog(
-                st.session_state.transcript_obj, 
-                st.session_state.visual_breakdown_obj
+            raw_blog_text = generate_production_blog(
+                st.session_state.transcript_obj,
+                st.session_state.visual_breakdown_obj,
+                uploaded_video.name
             )
-            
-            # Save the final blog post straight to the output directory
-            with open(BLOG_OUTPUT_PATH, "w", encoding="utf-8") as b_file:
-                b_file.write(st.session_state.final_blog)
+            st.session_state.final_blog = raw_blog_text
+
+            slug_match = re.search(r'slug:\s*"(.*?)"', raw_blog_text)
+            file_slug = slug_match.group(1) if slug_match else f"tutorial_{int(time.time())}"
+            unique_blog_filename = f"{datetime.now().strftime('%Y-%m-%d')}-{file_slug}.md"
+
+            final_blog_path = os.path.join(OUTPUT_DIR, unique_blog_filename)
+            with open(final_blog_path, "w", encoding="utf-8") as b_file:
+                b_file.write(raw_blog_text)
 
             st.write("Running Two-Stage Filtering Highlight Detection Engine...")
             # Automatically scans, validates, and runs 9:16 vertical center crops completely hands-free
@@ -119,9 +139,9 @@ if uploaded_video is not None:
                 visual_breakdown=st.session_state.visual_breakdown_obj,
                 layout_style=reel_style
             )
-            
+
             status.update(label="Phase 3 Complete: Content Generated & Reels Rendered!", state="complete")
-            
+
         st.session_state.pipeline_executed = True
         st.success("🎉 Multimodal Content Engine Processed All Layers Successfully with Zero Time Hallucinations!")
 

multimodal-engine/app/audio_processor.py

@@ -4,10 +4,21 @@ from dotenv import load_dotenv
 from google import genai
 from google.genai import types
 from utils import retry_with_backoff
-
 # Import the shared structural schema to fix your Streamlit ImportError
 from models import StructuredTranscript
 
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
+
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
+
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
 load_dotenv()
 
 def extract_audio_from_video(video_path, audio_output_path):
@@ -53,18 +64,18 @@ def transcribe_audio(audio_file_path) -> StructuredTranscript:
     
     print("🤖 Processing Speech-to-Text structured inference (Waiting for engine response)...")
 
-    def excute_call():
+    def execute_call():
         return client.models.generate_content(
             model="gemini-2.5-flash",
             contents=[upload_audio, prompt],
             config=types.GenerateContentConfig(
                 response_mime_type="application/json",
                 response_schema=StructuredTranscript,
-                temperature=0.0  # Zero out creativity to force strict transcription accuracy
+                temperature=0.0
             ),
         )
     try: 
-        response = retry_with_backoff(excute_call)
+        response = retry_with_backoff(execute_call)
         return response.parsed
         
     except Exception as e:  

multimodal-engine/app/clip_extractor.py

@@ -1,11 +1,10 @@
-import os
 from typing import List
 from pydantic import BaseModel, Field
 from google import genai
 from google.genai import types
 from audio_processor import StructuredTranscript
 from video_processor import ChronologicalVisualTimeline
-
+from utils import retry_with_backoff
 # Define a clean structural format for Stage 1 Candidate Output
 class CandidateHighlight(BaseModel):
     title: str = Field(description="A hook-driven viral title candidate.")
@@ -36,19 +35,18 @@ def stage_1_semantic_filter(audio_transcript: StructuredTranscript, visual_break
     """
 
     try:
-        response = client.models.generate_content(
-            model="gemini-2.5-flash",
-            contents=[prompt],
-            config=types.GenerateContentConfig(
-                response_mime_type="application/json",
-                response_schema=CandidateHighlightBatch,
-                temperature=0.2, # Low temperature to ensure time accuracy
-            ),
-        )
-        
-        parsed_response: CandidateHighlightBatch = response.parsed
-        print(f" Found {len(parsed_response.candidates)} candidate highlights for Stage 2 verification.")
-        return parsed_response.candidates
+        def execute_call():
+            return client.models.generate_content(
+                model="gemini-2.5-flash",
+                contents=[prompt],
+                config=types.GenerateContentConfig(
+                    response_mime_type="application/json",
+                    response_schema=CandidateHighlightBatch,
+                    temperature=0.2,
+                ),
+            )
+        response = retry_with_backoff(execute_call)
+        return response.parsed.candidates
 
     except Exception as e:
         print(f"🚨 Stage 1 Semantic Extraction Error: {str(e)}")

multimodal-engine/app/models.py

@@ -1,4 +1,3 @@
-# app/models.py
 from pydantic import BaseModel, Field
 from typing import List
 

multimodal-engine/app/reel_generator.py

@@ -7,7 +7,20 @@ from google import genai
 from google.genai import types
 from clip_extractor import CandidateHighlight
 from video_processor import generate_vertical_reel_clip
+from utils import retry_with_backoff
 
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
+
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
+
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
 # Define validation checks for our multi-modal confirmation layer
 class VisualVerificationReport(BaseModel):
     is_visually_engaging: bool = Field(description="True if the visual pacing, facial cues, or action sequences are high quality.")
@@ -22,14 +35,13 @@ def stage_2_visual_verification(video_path: str, candidates: List[CandidateHighl
     client = genai.Client()
     verified_production_clips = []
 
-    # Create a temporary directory to store raw clip slices
-    temp_dir = "../data/temp_verification_slices"
-    os.makedirs(temp_dir, exist_ok=True)
+    # Use the module-level resolved TEMP_DIR constant — no self-referencing os.path.join
+    os.makedirs(TEMP_DIR, exist_ok=True)
 
     for idx, candidate in enumerate(candidates):
         print(f" Processing verification step for candidate clip #{idx+1}: {candidate.title}")
         
-        temp_clip_path = os.path.join(temp_dir, f"candidate_slice_{idx}.mp4")
+        temp_clip_path = os.path.join(TEMP_DIR, f"candidate_slice_{idx}.mp4")
         duration = candidate.end_time - candidate.start_time
 
         # Quick FFmpeg slice to create a small evaluation file
@@ -60,23 +72,26 @@ def stage_2_visual_verification(video_path: str, candidates: List[CandidateHighl
         """
 
         try:
-            response = client.models.generate_content(
-                model="gemini-2.5-pro", # Multi-modal execution requires the Pro engine
-                contents=[uploaded_video, prompt],
-                config=types.GenerateContentConfig(
-                    response_mime_type="application/json",
-                    response_schema=VisualVerificationReport,
-                    temperature=0.1
-                ),
-            )
+            def execute_call():
+                return client.models.generate_content(
+                    model="gemini-2.5-pro",
+                    contents=[uploaded_video, prompt],
+                    config=types.GenerateContentConfig(
+                        response_mime_type="application/json",
+                        response_schema=VisualVerificationReport,
+                        temperature=0.1,
+                    ),
+                )
+            response = retry_with_backoff(execute_call)
             
             report: VisualVerificationReport = response.parsed
             
             if report.final_relevance_decision and report.audio_energy_score > 60:
                 print(f"    Passed Verification! Audio Energy Score: {report.audio_energy_score}/100")
-                # Update the target with our verified timestamps
-                candidate.start_time = report.adjusted_start_time + candidate.start_time
-                candidate.end_time = report.adjusted_end_time + candidate.start_time
+                
+                original_baseline_start = candidate.start_time
+                candidate.start_time = report.adjusted_start_time + original_baseline_start
+                candidate.end_time   = report.adjusted_end_time   + original_baseline_start
                 verified_production_clips.append(candidate)
             else:
                 print("    Failed Verification: Rejected due to flat visual energy levels.")

multimodal-engine/app/video_processor.py

@@ -9,16 +9,22 @@ from google.genai import types
 from dotenv import load_dotenv
 from openai import OpenAI
 from utils import retry_with_backoff
+from models import ChronologicalVisualTimeline, VideoFrameMoment
 
 load_dotenv()
 
-# Strict structures to link visual changes with audio timestamps cleanly
-class VideoFrameMoment(BaseModel):
-    timestamp_seconds: float = Field(description="The timestamp of this keyframe based on its position.")
-    visual_description: str = Field(description="Detailed summary of visual activity, slide content, text onscreen, or facial cues.")
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
 
-class ChronologicalVisualTimeline(BaseModel):
-    timeline: List[VideoFrameMoment]
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
+
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
 
 def extract_keyframes(video_path: str, keyframe_output_dir: str, interval_seconds: int = 5) -> int:
     """Extracts high-quality keyframes at precise uniform chronological markers."""
@@ -109,7 +115,7 @@ def analyze_scene_with_gemini(frame_dir: str, interval_seconds: int = 5) -> Opti
         Describe any visual changes, on-screen slide text, speaker facial actions, or object tracking details.
         """
 
-        def excute_call():
+        def execute_call():
             return client.models.generate_content(
                 model="gemini-2.5-flash",
                 contents=frame_uploaded + [prompt],
@@ -120,14 +126,23 @@ def analyze_scene_with_gemini(frame_dir: str, interval_seconds: int = 5) -> Opti
                 ),
             )
 
-        response = retry_with_backoff(excute_call)
+        response = retry_with_backoff(execute_call)
         print(" Gemini visual analysis execution step succeeded.")
         return response.parsed
     except Exception as gemini_error:
         print(f"🚨 Primary Gemini Cluster failed. Reason: {gemini_error}")
-        # Build prompt string for fallback handler matching our logic mapping structure
         fallback_prompt = f"Analyze these frames in sequential order. Provide a visual summary every {interval_seconds} seconds."
-        return run_openai_fallback(frame_paths, fallback_prompt)
+        raw_text_fallback = run_openai_fallback(frame_paths, fallback_prompt)
+        
+        from models import ChronologicalVisualTimeline, VideoFrameMoment
+        
+        fallback_desc = raw_text_fallback if raw_text_fallback else "Visual capture processing failure."
+        return ChronologicalVisualTimeline(timeline=[
+            VideoFrameMoment(
+                timestamp_seconds=float(i * interval_seconds),
+                visual_description=f"[OpenAI Fallback Data]: {fallback_desc}"
+            ) for i in range(1, len(frame_paths) + 1)
+        ])
     finally:
         # Guarantee cleanup loops execute under all load behaviors
         if frame_uploaded:
@@ -142,6 +157,17 @@ def generate_vertical_reel_clip(video_path: str, start_time: float, end_time: fl
     """Cuts and crops horizontal 16:9 video source files directly into a 9:16 vertical workspace canvas."""
     print(f"[3] Slicing and re-centering vertical layout from {start_time}s to {end_time}s")
     
+    if start_time < 0:
+        raise ValueError(f"❌ Pipeline Range Violation: start_time ({start_time}s) cannot be negative.")
+    if end_time <= start_time:
+        raise ValueError(f"❌ Pipeline Range Violation: end_time ({end_time}s) must occur after start_time ({start_time}s).")
+    if not os.path.exists(video_path):
+        raise FileNotFoundError(f"❌ Pipeline Resource Missing: Source target not located at {video_path}")
+        
+    output_directory = os.path.dirname(output_path)
+    if output_directory:
+        os.makedirs(output_directory, exist_ok=True)
+        
     if os.path.exists(output_path):
         os.remove(output_path)
 

multimodal-engine/app/workflow_engine.py

@@ -1,43 +1,74 @@
 import os
 import sys
+from datetime import datetime
 from dotenv import load_dotenv
 from google import genai
+from clip_extractor import stage_1_semantic_filter
+from reel_generator import stage_2_visual_verification
+
+APP_DIR = os.path.dirname(os.path.abspath(__file__))
+PROJECT_ROOT = os.path.abspath(os.path.join(APP_DIR, ".."))
+
+DATA_DIR = os.path.join(PROJECT_ROOT, "data")
+OUTPUT_DIR = os.path.join(PROJECT_ROOT, "output")
+CLIPS_DIR = os.path.join(OUTPUT_DIR, "clips")
+TEMP_DIR = os.path.join(DATA_DIR, "temp_verification_slices")
+
+# Guarantee that folder path hierarchies exist on disk before invoking downstream tools
+os.makedirs(DATA_DIR, exist_ok=True)
+os.makedirs(CLIPS_DIR, exist_ok=True)
+os.makedirs(TEMP_DIR, exist_ok=True)
 
 # Setup system environment routing parameters
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
 load_dotenv()
 
 # High-precision production component imports
-from audio_processor import extract_audio_from_video, transcribe_audio, save_transcript_todisk, StructuredTranscript
-from video_processor import extract_keyframes, analyze_scene_with_gemini, generate_vertical_reel_clip, ChronologicalVisualTimeline
-
+from audio_processor import extract_audio_from_video, transcribe_audio, save_transcript_todisk
+from video_processor import extract_keyframes, analyze_scene_with_gemini, generate_vertical_reel_clip
+from models import StructuredTranscript, ChronologicalVisualTimeline
 
-def generate_production_blog(audio_transcript: StructuredTranscript, visual_breakdown: ChronologicalVisualTimeline) -> str:
+def generate_production_blog(audio_transcript: StructuredTranscript, visual_breakdown: ChronologicalVisualTimeline, video_name: str) -> str:
     """Synthesizes structured visual mappings and text arrays into a markdown blog post."""
     print(" [3/4] Orchestrating final multimodal content synthesis via Gemini...")
     client = genai.Client()
+    current_date = datetime.now().strftime("%Y-%m-%d")
 
     # Pass the serialized clean string models to preserve structural hierarchy inside the token prompt space
     prompt = f"""
-    You are an expert technical content writer and developer documentation engineer.
-    
-    Synthesize these two multimodal timelines into a clear, detailed, step-by-step Technical Blog Post in Markdown:
+    You are an expert technical content writer and SEO documentation engineer.
     
+    I am providing you with two distinct data inputs extracted from the video file '{video_name}':
     1. TIMESTAMPED AUDIO TRANSCRIPT (JSON Mapping):
+    ---
     {audio_transcript.model_dump_json(indent=2)}
+    ---
     
     2. CHRONOLOGICAL VISUAL TIMELINE (JSON Mapping):
+    ---
     {visual_breakdown.model_dump_json(indent=2)}
+    ---
+    
+    CRITICAL STRUCTURE REQUIREMENT:
+    You MUST begin your response with exactly this standard YAML Front Matter block (do not add backticks, trailing syntax, or markdown wrappers around it):
+    ---
+    title: "Generate a catchy, high-impact, SEO-optimized title here"
+    date: "{current_date}"
+    tags: ["Artificial Intelligence", "Tutorial", "Tech Guide"]
+    category: "Technology"
+    author: "Multimodal AI Engine"
+    description: "Write a short, engaging 150-character meta description summary here for search previews."
+    slug: "generate-a-clean-url-friendly-lowercase-slug-here"
+    ---
     
-    STRUCTURE RULES:
-    - Add a catchy title at the top (#).
-    - Write a short introduction explaining what software/concept is being demonstrated.
-    - Break the content down into logical step-by-step sections using clear headings (##).
-    - Blend visual timeline actions smoothly with the spoken words so it reads like a cohesive tutorial.
-    - Highlight specific keyboard shortcuts, timestamps, or interface menus using code blocks or bold text.
-    - End with a summary conclusion.
+    Following the front-matter block, write the blog post document using these strict rules:
+    - Write a short introduction explaining the core software/concept demonstrated in the clip.
+    - Break the content down into logical sections using clear H2 (##) and H3 (###) headers.
+    - Incorporate structural elements like bullet points, summary tables, and bold code blocks cleanly.
+    - Blend the visual timeline shifts smoothly with the spoken words so it reads like a comprehensive, standalone web tutorial.
+    - Highlight keyboard shortcuts, timestamps, or interface menus mentioned on screen using bold text.
     
-    Do not add conversational commentary—return ONLY the markdown text blocks.
+    Do not add conversational commentary—return ONLY the markdown content.
     """
 
     try:
@@ -55,15 +86,10 @@ def run_integrated_pipeline(video_path: str):
     print("🚀 ----- Starting Integrated Multimodal Content Generation Pipeline -----")
 
     # Hardcoded configurations converted to strict, upgraded extensions
-    audio_output_path = "../data/extracted_audio.wav" 
-    frames_dir = "../data/extracted_frames"
-    blogs_output_path = "../output/how_multimodals_work_blog.md"
-    transcript_json_path = "../data/transcript.json"
-
-    if not os.path.exists("../output"):
-        os.makedirs("../output")
-    if not os.path.exists("../data"):
-        os.makedirs("../data")
+    audio_output_path = os.path.join(DATA_DIR, "extracted_audio.wav") 
+    frames_dir = os.path.join(DATA_DIR, "extracted_frames") 
+    blogs_output_path = os.path.join(OUTPUT_DIR, "how_multimodals_work_blog.md") 
+    transcript_json_path = os.path.join(DATA_DIR, "transcript.json") 
 
     # -------------------------------------------------------------
     # PHASE 1: Speech-To-Text Timeline Assembly
@@ -90,7 +116,7 @@ def run_integrated_pipeline(video_path: str):
     # -------------------------------------------------------------
     # PHASE 3: Content Marketing Synthesis (Long-Form Blog Asset)
     # -------------------------------------------------------------
-    final_blog_content = generate_production_blog(audio_transcript, visual_breakdown)
+    final_blog_content = generate_production_blog(audio_transcript, visual_breakdown, os.path.basename(video_path))
 
     if final_blog_content:
         with open(blogs_output_path, "w", encoding="utf-8") as f:
@@ -101,9 +127,6 @@ def run_integrated_pipeline(video_path: str):
     # PHASE 4: Short-Form Reel Extractor (Two-Stage Verification)
     # -------------------------------------------------------------
     print(" 🎬 [4/4] Activating Two-Stage Filtering Highlight Detection Routine...")
-    
-    from clip_extractor import stage_1_semantic_filter
-    from reel_generator import stage_2_visual_verification
 
     # 1. Run the Stage 1 text-based check
     candidate_clips = stage_1_semantic_filter(audio_transcript, visual_breakdown)
@@ -113,7 +136,7 @@ def run_integrated_pipeline(video_path: str):
     
     # 3. Render the verified shorts using our 9:16 vertical crop filter
     for idx, clip in enumerate(verified_clips):
-        reel_path = f"../output/viral_reel_{idx + 1}.mp4"
+        reel_path = os.path.join(CLIPS_DIR, f"viral_reel_{idx + 1}.mp4") 
         generate_vertical_reel_clip(video_path, clip.start_time, clip.end_time, reel_path)
 
     print("🏁 Pipeline run completed successfully.")

multimodal-engine/output/how_multimodals_work_blog.md

@@ -1,147 +1,33 @@
-# Demystifying Vector Embeddings: How AI Understands Meaning
+# Mending the Heart: Five Steps to Kintsugi Your Way Through Heartbreak
 
-Welcome to the world of modern Artificial Intelligence! Today, we're going to unravel one of its most fundamental concepts: **vector embeddings**. Have you ever wondered how a computer, a machine that only comprehends numbers, can grasp the nuanced meaning of a word like "king"? Or how it knows that "king" is closely related to "queen" but not to "apple"? The answer lies in a clever technique of transforming words into special lists of numbers, known as **vectors**.
+In the ancient Japanese art of Kintsugi, skilled artisans repair broken pottery using dust from gold or other precious metals. The beauty of Kintsugi lies not in hiding the damage, but in emphasizing the breaks and repairs, transforming them into a beautiful, integral part of the pottery's history. This philosophy teaches us that the breaks in an object's life can be highlighted to make it even more valuable and unique.
 
-## The Problem: Representing Words Digitally
+Similarly, when we experience heartbreak, we face a profound choice: will we forever view it as a wound, or will we transform it into art? Just as Kintsugi celebrates imperfection, we too can choose to see our emotional fractures as opportunities for growth and renewed strength. If you're navigating the pain of heartbreak, here are five practical steps inspired by the spirit of Kintsugi, designed to help you grow through your grief and emerge stronger.
 
-Before diving into the modern solution, let's understand the challenge.
+## 1. Jot in a Journal
 
-### The "Old Way": One-Hot Encoding
+Instead of replaying your heartbreak over and over in your head like a broken record, take the powerful step of recording your feelings in a journal. This act of transferring your thoughts and emotions to paper helps to create a crucial distance and perspective between you and your pain. As you capture your feelings, you'll begin to process them in a healthier way, preventing them from overwhelming your mind.
 
-**(Visual at 40.0s: 'Part one: The Old Way' title)**
+## 2. Get Out of Your Head and Into Your Body
 
-The simplest approach to represent words numerically is **one-hot encoding**. Imagine a small dictionary (our 'vocabulary') containing just a few words, each assigned a unique index.
+Heartbreak often manifests as a heavy weight on our chest, trapping us in a cycle of sadness and depression. Exercise is a potent remedy to lighten this load by encouraging movement and engagement with the physical world. It's more than just a physical activity; it's a mood enhancer, triggering your brain to release endorphins—natural feel-good chemicals. Beyond mood improvement, exercise promotes better sleep, which in turn boosts energy and vitality. This renewed energy can significantly shift your present reality, making it feel less bleak. Starting is often the hardest part, so make it easy: just put on your shoes and go for a walk in a beautiful place.
 
-**(Visual at 45.0s: 'One-Hot Encoding' slide with 'Vocabulary' and 'cat' mapping to '0 0 1 0')**
+## 3. Shift Your Thoughts to Thankfulness
 
-For example, if our vocabulary is:
-1.  Apple
-2.  Ball
-3.  Cat
-4.  Dog
+When heartbroken, our minds tend to fixate on the hurt, making it challenging to focus on anything else. One of the most accessible and impactful ways to change this mental pattern is by cultivating a gratitude list. Begin by identifying just three things you are genuinely grateful for. Don't just list them; elaborate with specific details. Think about the people, experiences, and small joys that have brought real meaning into your life. By consciously focusing on these positive elements, day by day, you will gradually feel ready to embrace a new beginning.
 
-To represent the word "cat", we'd create a vector with a length equal to our vocabulary size (in this case, four). The vector would have a '1' in the position corresponding to "cat"'s index, and '0's everywhere else. So, "cat" would become `[0, 0, 1, 0]`.
+## 4. Change Your Tune
 
-**(Visual at 50.0s: 'Vocabulary Mapping' showing King=0, Queen=1, Man=2, Woman=3. King=0 highlighted.)**
-**(Visual at 55.0s: 'Vocabulary' list (dog, cat, fish, bird) and 'cat' pointing to '[ zero, zero, one ,')**
-**(Visual at 65.0s: 'cat' mapping to '[1, 0, 0, 0]' and 'dog' mapping to '[0, 0, 0, 1]' highlighted.)**
+In moments of sorrow, it's common to seek solace in music that echoes our angst. However, music that matches a melancholic mood often amplifies our sadness and drains our energy. To truly heal, we need to lift our spirits, get energized, and find inspiration. Instead of dwelling in despair, consciously choose tunes that empower you and make you feel alive. Curate a playlist of uplifting songs that invigorate your spirit and remind you of your inner strength.
 
-Similarly, if "dog" was at the fourth position, its vector would be `[0, 0, 0, 1]`.
+## 5. Let Them Go
 
-### Why One-Hot Encoding Falls Short
+The most challenging aspect of a breakup isn't merely releasing the person; it's releasing all the possibilities, hopes, and dreams we projected onto that relationship. These projections, however, were never reality. It is crucial to acknowledge this distinction. Now is the time to release those idealized possibilities in favor of what can genuinely be your future. Holding onto the past will inevitably cause your present to slip by. By letting go, you create space for your real, authentic future to arrive.
 
-This method, while simple, presents two significant problems:
+## Finding Purpose and Embracing Wellness with Calm
 
-1.  **Inefficiency and Scalability (Visual at 70.0s: 'Problem one: HUGE & Inefficient')**
-    Real-world vocabularies contain tens of thousands of words, often **50,000 words or more**. This means each word would require a vector of 50,000 dimensions, with only one '1' and 49,999 '0's. This is incredibly inefficient in terms of storage and computation.
+Sometimes, underlying our emotional pain is a struggle with a lack of purpose. If you find yourself feeling immense pressure to know your life's direction, have lost interest and feel disconnected from your own life, or doubt your skills and abilities, hope is not lost. Having a sense of purpose provides vital guidance through life's inevitable ups and downs, lending structure to your daily existence.
 
-2.  **Lack of Semantic Meaning (Visual at 75.0s: 'Problem two: NO MEANING')**
-    More critically, one-hot vectors offer no inherent sense of meaning or relationship between words. Every word is equally "different" from every other word because their vectors are orthogonal (perpendicular). There's no mathematical way to discern that "cat" and "dog" are both animals, while "cat" and "bird" are also animals, but "cat" and "rock" are not. They're all just distinct points in a high-dimensional space.
+To support your journey towards healing and purpose, I've partnered with **Calm**, the leading app for mental health and wellness. The Calm app offers an extensive library of thousands of meditations, soothing songs designed to help you relax and focus, and immersive sleep stories to ensure a restful night's sleep. Additionally, you can find "The Daily Jay," a dedicated series where I share proven tools and techniques to improve your mindset and mental health in just a few minutes each day.
 
-    **(Visual at 80.0s: Words 'cat', 'dog', 'bird' shown as isolated points, illustrating no spatial relationship.)**
-
-## The Modern Solution: The Embedding Matrix
-
-**(Visual at 85.0s: 'Part two: The En' - beginning of 'The New Way')**
-
-Modern AI models overcome these limitations using an **embedding matrix**—a core component for representing words with meaning.
-
-### Defining the Embedding Space
-
-First, we define:
-*   **Vocabulary Size**: The total number of unique words (or "tokens") our model will understand. This could be 50,000 or more.
-*   **Embedding Dimension**: A much smaller, fixed number representing the length of each word's vector. Common dimensions are **300**, 768, or 1024. This dimension determines how much information each word vector can encode.
-
-**(Visual at 90.0s & 95.0s: 'Vocabulary' list (Size = 50,000) pointing to 'Embedding Dimension 300'.)**
-
-Our embedding matrix (often called a "lookup table") will therefore have:
-*   **Rows**: Equal to the vocabulary size (e.g., 50,000 rows, one for each word).
-*   **Columns**: Equal to the embedding dimension (e.g., 300 columns).
-
-**(Visual at 100.0s: 'Embedding Matrix' slide showing a large blue grid with 'fifty thousand rows' indicated by a yellow bracket.)**
-
-### Initialization
-
-At the very beginning of training an AI model, we fill this giant table with small, random numbers. These numbers, initially meaningless, will be adjusted during the training process.
-
-**(Visual at 105.0s: 'Embedding Matrix' grid with numerical values, some highlighted in red.)**
-
-### Assigning a Vector: A Simple Lookup
-
-So, how does a model get a vector for a specific word (or "token")? It's surprisingly simple: it's a direct lookup operation.
-
-**(Visual at 110.0s: 'queen' in a gray rectangle on the left, and a 'Vocabulary' list showing 'king ID: 41', 'queen ID: 42', 'prince ID: 43'.)**
-**(Visual at 115.0s: A gray rounded rectangle with 'ID: 42' next to a partially visible 'Embedding Matrix'.)**
-
-The model performs these steps:
-1.  It finds the unique **ID** corresponding to the word it needs to represent (e.g., "queen" might have ID: **42**).
-2.  It then goes to the embedding matrix and simply fetches the entire row of numbers associated with that ID.
-
-**(Visual at 120.0s: 'ID: 42' pointing to a highlighted yellow row in the 'Embedding Matrix', labeled 'Vector Embedding' below.)**
-
-This retrieved row of numbers is the word's vector embedding.
-
-## How the Vectors Get Smart: Learning Meaning Through Training
-
-**(Visual at 125.0s: 'Part Three: How The Vectors Get Smart' title.)**
-
-The magic happens during the model's training phase. These numbers aren't random forever; they are continuously refined.
-
-Consider a model learning to predict the next word in a sentence: "The queen wore a ___".
-
-**(Visual at 130.0s: Text 'The queen wore a ___' with a 'MODEL' button.)**
-
-1.  **Initial Prediction**: Based on its initial (random) embedding for "queen" and the other words, the model might incorrectly predict the next word is "shoe".
-
-    **(Visual at 135.0s: 'queen' vector pointing to 'MODEL' button, which points to a crossed-out 'shoe' with a red 'x'.)**
-
-2.  **Error Calculation and Adjustment**: The model calculates an error because "shoe" is not the correct next word (perhaps the correct word was "crown"). Using this error, the model then goes back and makes tiny adjustments to the numbers within the vector for "queen" (and potentially other words in the context).
-
-    **(Visual at 140.0s: 'Embedding Matrix' showing 'queen' highlighted, and 'Model Feedback' with a red 'x' and 'Correct: crown' with a green checkmark.)**
-
-3.  **Repetitive Learning**: This process is repeated millions, even billions, of times with vast amounts of text data from the internet. The model is constantly predicting, calculating errors, and adjusting the numerical values in its embedding matrix.
-
-    **(Visual at 145.0s: Dark background with faint grid lines and axis markers.)**
-
-### Semantic Closeness
-
-Over countless iterations, a remarkable phenomenon occurs:
-*   Words that frequently appear in similar contexts (like "king" and "queen", or "cat" and "kitten") will have their vectors nudged in similar directions.
-*   Words with dissimilar meanings (like "king" and "apple") will have their vectors pushed further apart.
-
-**(Visual at 150.0s: Scatter plot showing 'prince', 'queen', 'king' clustered together in yellow, and 'apple', 'orange' clustered in red.)**
-**(Visual at 155.0s: Scatter plot showing 'king', 'queen', 'prince' clustered in blue, and 'apple', 'orange' clustered in orange.)**
-
-Eventually, the vectors for related words end up mathematically "close" to each other in the high-dimensional space, effectively filling those random numbers with rich, semantic meaning. This allows AI models to understand relationships and context in a way that one-hot encoding never could.
-
-**(Visual at 160.0s: Various words like 'King', 'Queen', 'Prince', 'Throne', 'Crown' clustered, while fruits are elsewhere.)**
-
-## Recap: How an Embedding Model Works
-
-**(Visual at 165.0s: 'Token to Vector: A Recap' slide with four icons: 'Vocabulary & IDs', 'Embedding Matrix', 'Lookup', and 'Learn & Adjust'.)**
-
-Let's quickly recap the entire process:
-
-1.  **Vocabulary & IDs**: Every unique word is assigned a specific numerical ID.
-    **(Visual at 175.0s: Table with 'Token' and 'ID' columns, listing 'The' (0), 'quick' (1), 'brown' (2), 'fox' (3).)**
-
-2.  **Embedding Matrix Initialization**: A large embedding matrix is created, where each row corresponds to a word ID and contains a vector of random numbers. The number of columns determines the embedding dimension.
-    **(Visual at 180.0s: Table with 'Token' and 'ID' next to an 'Embedding Matrix' with values; 'quick' (ID 1) row highlighted.)**
-
-3.  **Vector Lookup**: To get a word's vector, the model simply uses the word's ID to fetch the corresponding row from the embedding matrix. This is a direct, efficient lookup.
-    **(Visual at 190.0s: 'Input Token ID: 3' pointing to a highlighted yellow row in the 'Embedding Matrix', labeled 'Vector Embedding'.)**
-
-4.  **Learning & Adjustment**: Through extensive training, the model constantly adjusts these numerical vectors based on predictions and errors. This process mathematically nudges related words closer together in the vector space, imbuing the numbers with semantic meaning.
-    **(Visual at 195.0s: Coordinate plane showing a red vector 'Cat' and a blue vector 'Kitten', with a circular arrow icon.)**
-
-## Conclusion
-
-**(Visual at 200.0s: 'Embedding Models: A Smart Lookup Table' title.)**
-
-In essence, an embedding model is a lookup table that starts off with random numbers and progressively gets smarter through experience. It learns to represent words as meaningful numerical vectors, allowing AI to understand language, perform complex tasks like translation and sentiment analysis, and interact with us in increasingly sophisticated ways.
-
-**(Visual at 205.0s: 'blackboard AI' logo displayed.)**
-
-If this explanation helped you demystify vector embeddings, please consider liking this blog post and subscribing for more straightforward breakdowns of complex AI topics.
-
-Thanks for reading!
+Embrace these steps, find your purpose, and remember that your journey through heartbreak can be a transformative process, akin to the golden repairs of Kintsugi, leaving you not broken, but beautifully restored.

multimodal-engine/pytest.ini

@@ -0,0 +1,4 @@
+[pytest]
+testpaths = test
+pythonpath = app
+addopts = -v --tb=short

multimodal-engine/requirements.txt

@@ -1,5 +1,5 @@
-streamlit>=1.35.0
-google-genai>=1.0.0
-openai>=1.30.0
-python-dotenv>=1.0.0
-pydantic>=2.0.0
+streamlit==1.57.0
+google-genai==1.68.0
+openai==2.37.0
+python-dotenv==1.0.1
+pydantic==2.12.5

multimodal-engine/test/test_agent_optimizer.py

@@ -0,0 +1,192 @@
+"""
+Tests for agent_optimizer.py
+Covers:
+- Layout style → render_mode routing logic (Blurred Stack vs AI Smart Crop)
+- Slug sanitisation logic used for output filenames
+- run_autonomous_editing_pipeline returns False when no highlights found
+- run_autonomous_editing_pipeline iterates and calls generate_vertical_reel_clip per highlight
+All Gemini API calls and FFmpeg calls are fully mocked.
+"""
+import os
+import sys
+import pytest
+from unittest.mock import patch, MagicMock, call
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../app")))
+
+from models import (
+    StructuredTranscript,
+    AudioSegment,
+    WordTimestamp,
+    ChronologicalVisualTimeline,
+    VideoFrameMoment,
+)
+
+
+# ─── Helpers ──────────────────────────────────────────────────────────────────
+
+def _make_transcript():
+    return StructuredTranscript(
+        segments=[
+            AudioSegment(
+                start_time=0.0,
+                end_time=10.0,
+                text="Hello world",
+                words=[WordTimestamp(word="Hello", start=0.0, end=0.5)],
+            )
+        ]
+    )
+
+def _make_timeline():
+    return ChronologicalVisualTimeline(
+        timeline=[VideoFrameMoment(timestamp_seconds=5.0, visual_description="Slide visible")]
+    )
+
+def _make_highlight(title="Great Hook", start=10.0, end=30.0, position="center"):
+    """Return a mock ProductionHighlight object."""
+    h = MagicMock()
+    h.hook_title = title
+    h.start_time = start
+    h.end_time = end
+    h.speaker_position = position
+    return h
+
+
+# ─── Slug sanitisation (inline logic extracted for testing) ───────────────────
+
+def _sanitise_slug(title: str) -> str:
+    """Mirrors the slug logic inside run_autonomous_editing_pipeline."""
+    return (
+        "".join(c for c in title if c.isalnum() or c in (" ", "_"))
+        .rstrip()
+        .replace(" ", "_")
+        .lower()
+    )
+
+class TestSlugSanitisation:
+    def test_spaces_become_underscores(self):
+        assert _sanitise_slug("Hello World") == "hello_world"
+
+    def test_special_chars_removed(self):
+        assert _sanitise_slug("Top 5 Tips! #viral") == "top_5_tips_viral"
+
+    def test_already_clean_title(self):
+        assert _sanitise_slug("clean_title") == "clean_title"
+
+    def test_empty_title(self):
+        assert _sanitise_slug("") == ""
+
+    def test_only_special_chars_becomes_empty(self):
+        assert _sanitise_slug("!!!") == ""
+
+
+# ─── Layout style → render mode routing ──────────────────────────────────────
+
+def _resolve_render_mode(layout_style: str, speaker_position: str) -> str:
+    """Mirrors the if/else in run_autonomous_editing_pipeline."""
+    if "Blurred Stack" in layout_style:
+        return "blurred"
+    return speaker_position
+
+class TestRenderModeRouting:
+    def test_blurred_stack_overrides_speaker_position(self):
+        assert _resolve_render_mode("Blurred Stack Mode (Presentation/Code)", "left") == "blurred"
+
+    def test_ai_smart_crop_uses_speaker_position_center(self):
+        assert _resolve_render_mode("AI Smart Face Crop (Podcast/Vlog)", "center") == "center"
+
+    def test_ai_smart_crop_uses_speaker_position_left(self):
+        assert _resolve_render_mode("AI Smart Face Crop (Podcast/Vlog)", "left") == "left"
+
+    def test_ai_smart_crop_uses_speaker_position_right(self):
+        assert _resolve_render_mode("AI Smart Face Crop (Podcast/Vlog)", "right") == "right"
+
+    def test_blurred_partial_match_still_resolves(self):
+        # Ensures substring matching works as coded
+        assert _resolve_render_mode("Blurred Stack", "right") == "blurred"
+
+
+# ─── run_autonomous_editing_pipeline ─────────────────────────────────────────
+
+class TestRunAutonomousEditingPipeline:
+
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously", return_value=[])
+    def test_returns_false_when_no_highlights(self, mock_discover, mock_render):
+        from agent_optimizer import run_autonomous_editing_pipeline
+        result = run_autonomous_editing_pipeline(
+            video_path="fake.mp4",
+            audio_transcript=_make_transcript(),
+            visual_breakdown=_make_timeline(),
+            layout_style="AI Smart Face Crop",
+        )
+        assert result is False
+        mock_render.assert_not_called()
+
+    @patch("agent_optimizer.os.makedirs")
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_returns_true_when_highlights_found(self, mock_discover, mock_render, mock_makedirs):
+        mock_discover.return_value = [_make_highlight("Good Clip", 5.0, 25.0, "center")]
+        from agent_optimizer import run_autonomous_editing_pipeline
+        result = run_autonomous_editing_pipeline(
+            video_path="fake.mp4",
+            audio_transcript=_make_transcript(),
+            visual_breakdown=_make_timeline(),
+            layout_style="AI Smart Face Crop",
+        )
+        assert result is True
+
+    @patch("agent_optimizer.os.makedirs")
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_calls_render_once_per_highlight(self, mock_discover, mock_render, mock_makedirs):
+        highlights = [
+            _make_highlight("Clip One", 5.0, 25.0, "center"),
+            _make_highlight("Clip Two", 40.0, 60.0, "left"),
+        ]
+        mock_discover.return_value = highlights
+        from agent_optimizer import run_autonomous_editing_pipeline
+        run_autonomous_editing_pipeline(
+            video_path="fake.mp4",
+            audio_transcript=_make_transcript(),
+            visual_breakdown=_make_timeline(),
+            layout_style="AI Smart Face Crop",
+        )
+        assert mock_render.call_count == 2
+
+    @patch("agent_optimizer.os.makedirs")
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_blurred_mode_overrides_render_mode(self, mock_discover, mock_render, mock_makedirs):
+        mock_discover.return_value = [_make_highlight("Demo", 0.0, 20.0, "right")]
+        from agent_optimizer import run_autonomous_editing_pipeline
+        run_autonomous_editing_pipeline(
+            video_path="fake.mp4",
+            audio_transcript=_make_transcript(),
+            visual_breakdown=_make_timeline(),
+            layout_style="Blurred Stack Mode",
+        )
+        _, kwargs = mock_render.call_args
+        assert kwargs["render_mode"] == "blurred"
+
+    @patch("agent_optimizer.os.makedirs")
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_ffmpeg_error_does_not_crash_pipeline(self, mock_discover, mock_render, mock_makedirs):
+        """A single clip render failure should not stop remaining clips."""
+        mock_discover.return_value = [
+            _make_highlight("Bad Clip", 0.0, 20.0, "center"),
+            _make_highlight("Good Clip", 30.0, 50.0, "center"),
+        ]
+        mock_render.side_effect = [Exception("ffmpeg failed"), None]
+        from agent_optimizer import run_autonomous_editing_pipeline
+        # Should not raise — the exception is caught internally
+        result = run_autonomous_editing_pipeline(
+            video_path="fake.mp4",
+            audio_transcript=_make_transcript(),
+            visual_breakdown=_make_timeline(),
+            layout_style="AI Smart Face Crop",
+        )
+        assert result is True
+        assert mock_render.call_count == 2

multimodal-engine/test/test_models.py

@@ -0,0 +1,162 @@
+"""
+Tests for Pydantic data models (models.py)
+These are pure unit tests — no API calls, no FFmpeg, no files needed.
+"""
+import pytest
+import sys
+import os
+
+# Allow imports from app/ without installing as a package
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../app")))
+
+from models import (
+    WordTimestamp,
+    AudioSegment,
+    StructuredTranscript,
+    VideoFrameMoment,
+    ChronologicalVisualTimeline,
+)
+
+
+# ─── WordTimestamp ────────────────────────────────────────────────────────────
+
+class TestWordTimestamp:
+    def test_valid_construction(self):
+        w = WordTimestamp(word="hello", start=0.0, end=0.45)
+        assert w.word == "hello"
+        assert w.start == 0.0
+        assert w.end == 0.45
+
+    def test_float_types_are_enforced(self):
+        # Pydantic coerces int → float automatically
+        w = WordTimestamp(word="hi", start=1, end=2)
+        assert isinstance(w.start, float)
+        assert isinstance(w.end, float)
+
+    def test_missing_field_raises(self):
+        with pytest.raises(Exception):
+            WordTimestamp(word="oops")  # start and end missing
+
+
+# ─── AudioSegment ─────────────────────────────────────────────────────────────
+
+class TestAudioSegment:
+    def _make_word(self, word="test", start=0.0, end=0.5):
+        return WordTimestamp(word=word, start=start, end=end)
+
+    def test_valid_construction(self):
+        seg = AudioSegment(
+            start_time=0.0,
+            end_time=5.0,
+            text="Hello world",
+            words=[self._make_word()],
+        )
+        assert seg.text == "Hello world"
+        assert len(seg.words) == 1
+
+    def test_empty_words_list_is_valid(self):
+        seg = AudioSegment(start_time=0.0, end_time=1.0, text="ok", words=[])
+        assert seg.words == []
+
+    def test_multiple_words(self):
+        words = [
+            self._make_word("Hi", 0.0, 0.3),
+            self._make_word("there", 0.3, 0.7),
+        ]
+        seg = AudioSegment(start_time=0.0, end_time=1.0, text="Hi there", words=words)
+        assert len(seg.words) == 2
+        assert seg.words[1].word == "there"
+
+
+# ─── StructuredTranscript ─────────────────────────────────────────────────────
+
+class TestStructuredTranscript:
+    def _make_segment(self, start=0.0, end=5.0, text="sample"):
+        return AudioSegment(
+            start_time=start,
+            end_time=end,
+            text=text,
+            words=[WordTimestamp(word=text, start=start, end=end)],
+        )
+
+    def test_single_segment(self):
+        t = StructuredTranscript(segments=[self._make_segment()])
+        assert len(t.segments) == 1
+
+    def test_multiple_segments(self):
+        t = StructuredTranscript(
+            segments=[
+                self._make_segment(0.0, 5.0, "first"),
+                self._make_segment(5.0, 10.0, "second"),
+            ]
+        )
+        assert len(t.segments) == 2
+        assert t.segments[0].text == "first"
+        assert t.segments[1].text == "second"
+
+    def test_empty_segments_is_valid(self):
+        t = StructuredTranscript(segments=[])
+        assert t.segments == []
+
+    def test_serialise_to_json_and_back(self):
+        original = StructuredTranscript(segments=[self._make_segment()])
+        json_str = original.model_dump_json()
+        restored = StructuredTranscript.model_validate_json(json_str)
+        assert restored.segments[0].text == original.segments[0].text
+
+    def test_float_timestamps_preserved_in_json(self):
+        seg = self._make_segment(start=12.5, end=20.75)
+        t = StructuredTranscript(segments=[seg])
+        data = t.model_dump()
+        assert data["segments"][0]["start_time"] == 12.5
+        assert data["segments"][0]["end_time"] == 20.75
+
+
+# ─── VideoFrameMoment ─────────────────────────────────────────────────────────
+
+class TestVideoFrameMoment:
+    def test_valid_construction(self):
+        frame = VideoFrameMoment(
+            timestamp_seconds=10.0,
+            visual_description="Speaker gestures at whiteboard.",
+        )
+        assert frame.timestamp_seconds == 10.0
+
+    def test_zero_timestamp_is_valid(self):
+        frame = VideoFrameMoment(timestamp_seconds=0.0, visual_description="Intro slide.")
+        assert frame.timestamp_seconds == 0.0
+
+    def test_missing_description_raises(self):
+        with pytest.raises(Exception):
+            VideoFrameMoment(timestamp_seconds=5.0)
+
+
+# ─── ChronologicalVisualTimeline ──────────────────────────────────────────────
+
+class TestChronologicalVisualTimeline:
+    def _make_frame(self, ts=5.0, desc="Frame description"):
+        return VideoFrameMoment(timestamp_seconds=ts, visual_description=desc)
+
+    def test_single_frame(self):
+        tl = ChronologicalVisualTimeline(timeline=[self._make_frame()])
+        assert len(tl.timeline) == 1
+
+    def test_ordered_timestamps(self):
+        tl = ChronologicalVisualTimeline(
+            timeline=[
+                self._make_frame(5.0, "frame 1"),
+                self._make_frame(10.0, "frame 2"),
+                self._make_frame(15.0, "frame 3"),
+            ]
+        )
+        timestamps = [f.timestamp_seconds for f in tl.timeline]
+        assert timestamps == sorted(timestamps), "Timeline should be in ascending order"
+
+    def test_serialise_roundtrip(self):
+        original = ChronologicalVisualTimeline(timeline=[self._make_frame(25.0, "desk shot")])
+        restored = ChronologicalVisualTimeline.model_validate_json(original.model_dump_json())
+        assert restored.timeline[0].timestamp_seconds == 25.0
+
+    def test_empty_timeline_is_valid(self):
+        tl = ChronologicalVisualTimeline(timeline=[])
+        assert tl.timeline == []

multimodal-engine/test/test_pipeline_integration.py

@@ -0,0 +1,449 @@
+"""
+Integration tests for the full 4-phase pipeline.
+
+All external boundaries (Gemini, OpenAI, FFmpeg, disk I/O) are mocked so these
+tests run without API keys, video files, or FFmpeg installed.
+
+They verify that the phases wire together correctly:
+  Phase 1 → audio extraction + transcription  → StructuredTranscript
+  Phase 2 → keyframe extraction + VLM analysis → ChronologicalVisualTimeline
+  Phase 3 → blog synthesis                     → Markdown string saved to disk
+  Phase 4 → highlight detection + reel render  → MP4 files written to output/clips
+"""
+
+import os
+import sys
+import json
+import pytest
+from unittest.mock import patch, MagicMock, call
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../app")))
+
+from models import (
+    StructuredTranscript,
+    AudioSegment,
+    WordTimestamp,
+    ChronologicalVisualTimeline,
+    VideoFrameMoment,
+)
+
+
+# ─── Reusable fixtures ────────────────────────────────────────────────────────
+
+@pytest.fixture
+def sample_transcript():
+    return StructuredTranscript(
+        segments=[
+            AudioSegment(
+                start_time=0.0,
+                end_time=15.0,
+                text="Welcome to the tutorial. Today we cover embeddings.",
+                words=[
+                    WordTimestamp(word="Welcome", start=0.0, end=0.5),
+                    WordTimestamp(word="embeddings", start=14.0, end=14.8),
+                ],
+            ),
+            AudioSegment(
+                start_time=15.0,
+                end_time=45.0,
+                text="Vectors allow semantic similarity search across large datasets.",
+                words=[
+                    WordTimestamp(word="Vectors", start=15.0, end=15.6),
+                    WordTimestamp(word="datasets", start=44.0, end=44.9),
+                ],
+            ),
+        ]
+    )
+
+
+@pytest.fixture
+def sample_timeline():
+    return ChronologicalVisualTimeline(
+        timeline=[
+            VideoFrameMoment(timestamp_seconds=5.0,  visual_description="Title slide: AI Embeddings"),
+            VideoFrameMoment(timestamp_seconds=10.0, visual_description="Speaker at whiteboard"),
+            VideoFrameMoment(timestamp_seconds=15.0, visual_description="Code editor visible"),
+            VideoFrameMoment(timestamp_seconds=20.0, visual_description="Vector diagram on screen"),
+        ]
+    )
+
+
+# ─── Phase 1 integration: audio extraction → transcription ───────────────────
+
+class TestPhase1AudioTranscription:
+
+    @patch("audio_processor.genai.Client")
+    @patch("audio_processor.subprocess.run")
+    def test_extract_then_transcribe_returns_structured_transcript(
+        self, mock_run, mock_client_cls, tmp_path, sample_transcript
+    ):
+        """FFmpeg succeeds → Gemini returns parsed StructuredTranscript."""
+        mock_run.return_value = MagicMock(returncode=0)
+
+        # Build a mock Gemini client that returns sample_transcript via .parsed
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_upload = MagicMock()
+        mock_upload.name = "files/test-audio-001"
+        mock_client.files.upload.return_value = mock_upload
+        mock_response = MagicMock()
+        mock_response.parsed = sample_transcript
+        mock_client.models.generate_content.return_value = mock_response
+
+        from audio_processor import extract_audio_from_video, transcribe_audio
+
+        video_path = str(tmp_path / "video.mp4")
+        audio_path = str(tmp_path / "audio.wav")
+
+        # Create a dummy video file so FFmpeg mock has a real path
+        open(video_path, "wb").close()
+
+        extract_audio_from_video(video_path, audio_path)
+
+        # Create the audio file so transcribe_audio can upload it
+        open(audio_path, "wb").close()
+        result = transcribe_audio(audio_path)
+
+        assert isinstance(result, StructuredTranscript)
+        assert len(result.segments) == 2
+        assert result.segments[0].text == "Welcome to the tutorial. Today we cover embeddings."
+
+    @patch("audio_processor.genai.Client")
+    @patch("audio_processor.subprocess.run")
+    def test_transcription_none_parsed_raises(self, mock_run, mock_client_cls, tmp_path):
+        """If Gemini returns None for .parsed the exception propagates cleanly."""
+        mock_run.return_value = MagicMock(returncode=0)
+
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_upload = MagicMock()
+        mock_upload.name = "files/test-audio-002"
+        mock_client.files.upload.return_value = mock_upload
+        mock_response = MagicMock()
+        mock_response.parsed = None
+        mock_client.models.generate_content.return_value = mock_response
+
+        from audio_processor import transcribe_audio
+
+        audio_path = str(tmp_path / "audio.wav")
+        open(audio_path, "wb").close()
+
+        # Should not crash silently — caller must handle None
+        result = transcribe_audio(audio_path)
+        assert result is None  # pipeline callers are responsible for the guard
+
+
+# ─── Phase 2 integration: keyframe extraction → scene analysis ───────────────
+
+class TestPhase2VisualTimeline:
+
+    @patch("video_processor.genai.Client")
+    @patch("video_processor.subprocess.run")
+    def test_extract_then_analyze_returns_timeline(
+        self, mock_run, mock_client_cls, tmp_path, sample_timeline
+    ):
+        """FFmpeg succeeds → Gemini returns a ChronologicalVisualTimeline."""
+        mock_run.return_value = MagicMock(returncode=0)
+
+        # Create fake JPEG frames so glob finds them
+        frames_dir = tmp_path / "frames"
+        frames_dir.mkdir()
+        for i in range(1, 5):
+            (frames_dir / f"keyframe_{i:04d}.jpg").write_bytes(b"\xff\xd8\xff")
+
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_upload = MagicMock()
+        mock_upload.name = "files/frame-001"
+        mock_client.files.upload.return_value = mock_upload
+        mock_response = MagicMock()
+        mock_response.parsed = sample_timeline
+        mock_client.models.generate_content.return_value = mock_response
+
+        from video_processor import analyze_scene_with_gemini
+
+        result = analyze_scene_with_gemini(str(frames_dir), interval_seconds=5)
+
+        assert isinstance(result, ChronologicalVisualTimeline)
+        assert len(result.timeline) == 4
+        assert result.timeline[0].timestamp_seconds == 5.0
+
+    @patch("video_processor.run_openai_fallback")
+    @patch("video_processor.genai.Client")
+    def test_gemini_failure_falls_back_to_openai_timeline(
+        self, mock_client_cls, mock_fallback, tmp_path
+    ):
+        """When Gemini raises, the OpenAI fallback returns a valid timeline object."""
+        frames_dir = tmp_path / "frames"
+        frames_dir.mkdir()
+        for i in range(1, 3):
+            (frames_dir / f"keyframe_{i:04d}.jpg").write_bytes(b"\xff\xd8\xff")
+
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_client.files.upload.side_effect = RuntimeError("Gemini upload failed")
+        mock_fallback.return_value = "Whiteboard with diagrams visible on screen."
+
+        from video_processor import analyze_scene_with_gemini
+
+        result = analyze_scene_with_gemini(str(frames_dir), interval_seconds=5)
+
+        # Must return a typed ChronologicalVisualTimeline — not a raw string
+        assert isinstance(result, ChronologicalVisualTimeline)
+        assert len(result.timeline) == 2
+        assert "[OpenAI Fallback Data]" in result.timeline[0].visual_description
+
+    @patch("video_processor.run_openai_fallback")
+    @patch("video_processor.genai.Client")
+    def test_both_providers_fail_returns_placeholder_timeline(
+        self, mock_client_cls, mock_fallback, tmp_path
+    ):
+        """If both Gemini and OpenAI fail, a placeholder timeline is still returned."""
+        frames_dir = tmp_path / "frames"
+        frames_dir.mkdir()
+        (frames_dir / "keyframe_0001.jpg").write_bytes(b"\xff\xd8\xff")
+
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_client.files.upload.side_effect = RuntimeError("Gemini down")
+        mock_fallback.return_value = None  # OpenAI also fails
+
+        from video_processor import analyze_scene_with_gemini
+
+        result = analyze_scene_with_gemini(str(frames_dir), interval_seconds=5)
+
+        assert isinstance(result, ChronologicalVisualTimeline)
+        assert result.timeline[0].visual_description == "[OpenAI Fallback Data]: Visual capture processing failure."
+
+
+# ─── Phase 3 integration: blog synthesis ─────────────────────────────────────
+
+class TestPhase3BlogSynthesis:
+
+    @patch("workflow_engine.genai.Client")
+    def test_generate_blog_embeds_transcript_in_prompt(
+        self, mock_client_cls, sample_transcript, sample_timeline, tmp_path
+    ):
+        """Verify the transcript JSON is actually injected into the Gemini prompt."""
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_response = MagicMock()
+        mock_response.text = (
+            '---\ntitle: "Test"\nslug: "test-slug"\n---\n\n## Intro\nSome content.'
+        )
+        mock_client.models.generate_content.return_value = mock_response
+
+        from workflow_engine import generate_production_blog
+
+        result = generate_production_blog(sample_transcript, sample_timeline, "tutorial.mp4")
+
+        # The prompt sent to Gemini must contain the serialised transcript data
+        call_args = mock_client.models.generate_content.call_args
+        prompt_sent = call_args[1]["contents"][0] if call_args[1] else call_args[0][1][0]
+        assert "Welcome to the tutorial" in prompt_sent or "segments" in prompt_sent
+
+    @patch("workflow_engine.genai.Client")
+    def test_generate_blog_returns_string(
+        self, mock_client_cls, sample_transcript, sample_timeline
+    ):
+        mock_client = MagicMock()
+        mock_client_cls.return_value = mock_client
+        mock_client.models.generate_content.return_value.text = "# Blog\n\nContent here."
+
+        from workflow_engine import generate_production_blog
+
+        result = generate_production_blog(sample_transcript, sample_timeline, "video.mp4")
+
+        assert isinstance(result, str)
+        assert len(result) > 0
+
+
+# ─── Phase 4 integration: highlight detection → reel rendering ───────────────
+
+class TestPhase4ReelPipeline:
+
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_pipeline_calls_render_for_each_highlight(
+        self, mock_discover, mock_render, sample_transcript, sample_timeline
+    ):
+        """Each discovered highlight must produce exactly one render call."""
+        h1 = MagicMock()
+        h1.hook_title = "Why Embeddings Matter"
+        h1.start_time = 10.0
+        h1.end_time = 35.0
+        h1.speaker_position = "center"
+
+        h2 = MagicMock()
+        h2.hook_title = "Vector Search Demo"
+        h2.start_time = 40.0
+        h2.end_time = 65.0
+        h2.speaker_position = "left"
+
+        mock_discover.return_value = [h1, h2]
+
+        from agent_optimizer import run_autonomous_editing_pipeline
+
+        result = run_autonomous_editing_pipeline(
+            video_path="fake_video.mp4",
+            audio_transcript=sample_transcript,
+            visual_breakdown=sample_timeline,
+            layout_style="AI Smart Face Crop (Podcast/Vlog)",
+        )
+
+        assert result is True
+        assert mock_render.call_count == 2
+
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    def test_blurred_mode_overrides_all_speaker_positions(
+        self, mock_discover, mock_render, sample_transcript, sample_timeline
+    ):
+        """Blurred Stack layout must ignore speaker_position for every clip."""
+        for pos in ["left", "center", "right"]:
+            mock_render.reset_mock()
+            h = MagicMock()
+            h.hook_title = "Test Clip"
+            h.start_time = 5.0
+            h.end_time = 25.0
+            h.speaker_position = pos
+            mock_discover.return_value = [h]
+
+            from agent_optimizer import run_autonomous_editing_pipeline
+
+            run_autonomous_editing_pipeline(
+                video_path="fake_video.mp4",
+                audio_transcript=sample_transcript,
+                visual_breakdown=sample_timeline,
+                layout_style="Blurred Stack Mode (Presentation/Code)",
+            )
+
+            _, kwargs = mock_render.call_args
+            assert kwargs["render_mode"] == "blurred", (
+                f"Expected 'blurred' for speaker_position='{pos}', got '{kwargs['render_mode']}'"
+            )
+
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously", return_value=[])
+    def test_empty_highlights_returns_false_no_render(
+        self, mock_discover, mock_render, sample_transcript, sample_timeline
+    ):
+        from agent_optimizer import run_autonomous_editing_pipeline
+
+        result = run_autonomous_editing_pipeline(
+            video_path="fake_video.mp4",
+            audio_transcript=sample_transcript,
+            visual_breakdown=sample_timeline,
+            layout_style="AI Smart Face Crop",
+        )
+
+        assert result is False
+        mock_render.assert_not_called()
+
+
+# ─── Full pipeline smoke test ─────────────────────────────────────────────────
+
+class TestFullPipelineSmoke:
+
+    @patch("agent_optimizer.generate_vertical_reel_clip")
+    @patch("agent_optimizer.discover_highlights_autonomously")
+    @patch("workflow_engine.genai.Client")
+    @patch("video_processor.genai.Client")
+    @patch("audio_processor.genai.Client")
+    @patch("video_processor.subprocess.run")
+    @patch("audio_processor.subprocess.run")
+    def test_all_4_phases_complete_without_error(
+        self,
+        mock_audio_run,
+        mock_video_run,
+        mock_audio_client_cls,
+        mock_video_client_cls,
+        mock_blog_client_cls,
+        mock_discover,
+        mock_render,
+        tmp_path,
+        sample_transcript,
+        sample_timeline,
+    ):
+        """
+        Smoke test: mock every external call and confirm all 4 phases
+        run end-to-end without raising any exception.
+        """
+        # --- FFmpeg mocks ---
+        mock_audio_run.return_value = MagicMock(returncode=0)
+        mock_video_run.return_value = MagicMock(returncode=0)
+
+        # --- Phase 1: transcription ---
+        audio_client = MagicMock()
+        mock_audio_client_cls.return_value = audio_client
+        mock_audio_upload = MagicMock()
+        mock_audio_upload.name = "files/audio-smoke"
+        audio_client.files.upload.return_value = mock_audio_upload
+        audio_response = MagicMock()
+        audio_response.parsed = sample_transcript
+        audio_client.models.generate_content.return_value = audio_response
+
+        # --- Phase 2: scene analysis (use pre-existing frames dir) ---
+        frames_dir = tmp_path / "frames"
+        frames_dir.mkdir()
+        for i in range(1, 4):
+            (frames_dir / f"keyframe_{i:04d}.jpg").write_bytes(b"\xff\xd8\xff")
+
+        video_client = MagicMock()
+        mock_video_client_cls.return_value = video_client
+        mock_frame_upload = MagicMock()
+        mock_frame_upload.name = "files/frame-smoke"
+        video_client.files.upload.return_value = mock_frame_upload
+        video_response = MagicMock()
+        video_response.parsed = sample_timeline
+        video_client.models.generate_content.return_value = video_response
+
+        # --- Phase 3: blog synthesis ---
+        blog_client = MagicMock()
+        mock_blog_client_cls.return_value = blog_client
+        blog_response = MagicMock()
+        blog_response.text = '---\ntitle: "Smoke Test"\nslug: "smoke-test"\n---\n\n## Content\nOK.'
+        blog_client.models.generate_content.return_value = blog_response
+
+        # --- Phase 4: highlight detection + render ---
+        highlight = MagicMock()
+        highlight.hook_title = "Key Insight"
+        highlight.start_time = 5.0
+        highlight.end_time = 25.0
+        highlight.speaker_position = "center"
+        mock_discover.return_value = [highlight]
+
+        # --- Run the full pipeline ---
+        audio_path = str(tmp_path / "audio.wav")
+        open(audio_path, "wb").close()
+
+        video_path = str(tmp_path / "video.mp4")
+        open(video_path, "wb").close()
+
+        from audio_processor import extract_audio_from_video, transcribe_audio
+        from video_processor import analyze_scene_with_gemini
+        from workflow_engine import generate_production_blog
+        from agent_optimizer import run_autonomous_editing_pipeline
+
+        # Phase 1
+        extract_audio_from_video(video_path, audio_path)
+        transcript = transcribe_audio(audio_path)
+        assert transcript is not None
+
+        # Phase 2
+        timeline = analyze_scene_with_gemini(str(frames_dir), interval_seconds=5)
+        assert timeline is not None
+
+        # Phase 3
+        blog = generate_production_blog(transcript, timeline, "video.mp4")
+        assert isinstance(blog, str) and len(blog) > 0
+
+        # Phase 4
+        result = run_autonomous_editing_pipeline(
+            video_path=video_path,
+            audio_transcript=transcript,
+            visual_breakdown=timeline,
+            layout_style="AI Smart Face Crop",
+        )
+        assert result is True
+        assert mock_render.call_count == 1

multimodal-engine/test/test_utils.py

@@ -0,0 +1,129 @@
+"""
+Tests for retry_with_backoff (utils.py)
+All tests are pure unit tests — no real API calls are made.
+APIError is faked via a real subclass so isinstance() checks inside utils.py pass.
+"""
+import pytest
+import sys
+import os
+from unittest.mock import patch, MagicMock
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../app")))
+
+from google.genai.errors import APIError
+from utils import retry_with_backoff
+
+
+class FakeAPIError(APIError):
+    """
+    A real subclass of APIError so isinstance(err, APIError) returns True.
+    APIError.__init__ requires a specific signature we bypass here.
+    """
+    def __init__(self, code: int):
+        # Skip the parent __init__ to avoid needing a full Response object
+        self.code = code
+        self.message = f"Fake API error {code}"
+
+    def __str__(self):
+        return self.message
+
+
+def _make_api_error(code: int) -> FakeAPIError:
+    """Return a real APIError subclass instance with the given HTTP status code."""
+    return FakeAPIError(code)
+
+
+class TestRetryWithBackoff:
+
+    # ── Success path ──────────────────────────────────────────────────────────
+
+    def test_returns_value_on_first_success(self):
+        func = MagicMock(return_value="ok")
+        result = retry_with_backoff(func, max_retries=3, initial_delay=0)
+        assert result == "ok"
+        func.assert_called_once()
+
+    # ── Retry on retriable codes ───────────────────────────────────────────────
+
+    @patch("utils.time.sleep")
+    def test_retries_on_429_then_succeeds(self, mock_sleep):
+        err = _make_api_error(429)
+        func = MagicMock(side_effect=[err, err, "recovered"])
+        result = retry_with_backoff(func, max_retries=5, initial_delay=0, backoff_factor=2)
+        assert result == "recovered"
+        assert func.call_count == 3
+
+    @patch("utils.time.sleep")
+    def test_retries_on_503_then_succeeds(self, mock_sleep):
+        err = _make_api_error(503)
+        func = MagicMock(side_effect=[err, "ok"])
+        result = retry_with_backoff(func, max_retries=3, initial_delay=0)
+        assert result == "ok"
+        assert func.call_count == 2
+
+    # ── Exhausted retries ─────────────────────────────────────────────────────
+
+    @patch("utils.time.sleep")
+    def test_raises_after_max_retries_exhausted(self, mock_sleep):
+        err = _make_api_error(429)
+        func = MagicMock(side_effect=err)
+        with pytest.raises(FakeAPIError):
+            retry_with_backoff(func, max_retries=3, initial_delay=0)
+        assert func.call_count == 3
+
+    # ── Non-retriable codes raise immediately ─────────────────────────────────
+
+    @patch("utils.time.sleep")
+    def test_raises_immediately_on_non_retriable_code(self, mock_sleep):
+        err = _make_api_error(400)  # Bad request — not retriable
+        func = MagicMock(side_effect=err)
+        with pytest.raises(FakeAPIError):
+            retry_with_backoff(func, max_retries=5, initial_delay=0)
+        # Should not retry — called exactly once
+        assert func.call_count == 1
+
+    @patch("utils.time.sleep")
+    def test_raises_immediately_on_401(self, mock_sleep):
+        err = _make_api_error(401)
+        func = MagicMock(side_effect=err)
+        with pytest.raises(FakeAPIError):
+            retry_with_backoff(func, max_retries=5, initial_delay=0)
+        assert func.call_count == 1
+
+    # ── Non-API exceptions propagate immediately ──────────────────────────────
+
+    def test_non_api_exception_propagates_immediately(self):
+        func = MagicMock(side_effect=ValueError("bad input"))
+        with pytest.raises(ValueError, match="bad input"):
+            retry_with_backoff(func, max_retries=5, initial_delay=0)
+        func.assert_called_once()
+
+    def test_runtime_error_propagates_immediately(self):
+        func = MagicMock(side_effect=RuntimeError("crash"))
+        with pytest.raises(RuntimeError):
+            retry_with_backoff(func, max_retries=5, initial_delay=0)
+        func.assert_called_once()
+
+    # ── Sleep is actually called between retries ───────────────────────────────
+
+    @patch("utils.time.sleep")
+    def test_sleep_is_called_between_retries(self, mock_sleep):
+        err = _make_api_error(429)
+        func = MagicMock(side_effect=[err, err, "ok"])
+        retry_with_backoff(func, max_retries=5, initial_delay=1, backoff_factor=2)
+        # Should sleep twice (after attempt 1 and attempt 2)
+        assert mock_sleep.call_count == 2
+
+    # ── Backoff delay grows with each retry ───────────────────────────────────
+
+    @patch("utils.random.uniform", return_value=0.0)   # Remove jitter noise
+    @patch("utils.time.sleep")
+    def test_delay_doubles_with_backoff_factor(self, mock_sleep, mock_random):
+        err = _make_api_error(429)
+        func = MagicMock(side_effect=[err, err, err, "ok"])
+        retry_with_backoff(func, max_retries=5, initial_delay=1, backoff_factor=2)
+        sleep_calls = [c.args[0] for c in mock_sleep.call_args_list]
+        # Delays should be 1, 2, 4 (doubling each time, jitter zeroed out)
+        assert sleep_calls[0] == pytest.approx(1.0, abs=0.01)
+        assert sleep_calls[1] == pytest.approx(2.0, abs=0.01)
+        assert sleep_calls[2] == pytest.approx(4.0, abs=0.01)

multimodal-engine/test/test_video_processor.py

@@ -0,0 +1,189 @@
+"""
+Tests for video_processor.py
+- extract_keyframes: tested with a real temp directory + mocked subprocess
+- encode_image_to_base64: tested with a real temp file
+- generate_vertical_reel_clip: filter-chain logic tested without running FFmpeg
+- analyze_scene_with_gemini / run_openai_fallback: patched at the boundary
+"""
+import os
+import sys
+import glob
+import base64
+import tempfile
+import subprocess
+import pytest
+from unittest.mock import patch, MagicMock, call
+
+sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../app")))
+
+import video_processor
+from video_processor import (
+    encode_image_to_base64,
+    extract_keyframes,
+    generate_vertical_reel_clip,
+)
+
+
+# ─── encode_image_to_base64 ───────────────────────────────────────────────────
+
+class TestEncodeImageToBase64:
+    def test_encodes_real_file(self, tmp_path):
+        img = tmp_path / "frame.jpg"
+        img.write_bytes(b"\xff\xd8\xff")   # minimal JPEG header bytes
+        result = encode_image_to_base64(str(img))
+        # Result must be a valid base64 string that decodes back to our bytes
+        assert base64.b64decode(result) == b"\xff\xd8\xff"
+
+    def test_returns_string(self, tmp_path):
+        img = tmp_path / "frame.jpg"
+        img.write_bytes(b"abc")
+        assert isinstance(encode_image_to_base64(str(img)), str)
+
+    def test_missing_file_raises(self):
+        with pytest.raises(FileNotFoundError):
+            encode_image_to_base64("/nonexistent/path/frame.jpg")
+
+
+# ─── extract_keyframes ────────────────────────────────────────────────────────
+
+class TestExtractKeyframes:
+
+    @patch("video_processor.subprocess.run")
+    def test_creates_output_dir_if_missing(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        output_dir = tmp_path / "frames"
+        extract_keyframes("fake.mp4", str(output_dir), interval_seconds=5)
+        assert output_dir.exists()
+
+    @patch("video_processor.subprocess.run")
+    def test_clears_existing_jpgs_before_extraction(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        output_dir = tmp_path / "frames"
+        output_dir.mkdir()
+        # Pre-populate with stale files
+        (output_dir / "keyframe_0001.jpg").write_bytes(b"old")
+        (output_dir / "keyframe_0002.jpg").write_bytes(b"old")
+        extract_keyframes("fake.mp4", str(output_dir), interval_seconds=5)
+        # After the call (before ffmpeg actually writes) the old files are gone
+        assert len(list(output_dir.glob("*.jpg"))) == 0
+
+    @patch("video_processor.subprocess.run")
+    def test_ffmpeg_command_uses_correct_fps_filter(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        output_dir = tmp_path / "frames"
+        extract_keyframes("fake.mp4", str(output_dir), interval_seconds=10)
+        cmd = mock_run.call_args[0][0]
+        assert "fps=1/10" in cmd
+
+    @patch("video_processor.subprocess.run")
+    def test_ffmpeg_uses_high_quality_jpeg(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        output_dir = tmp_path / "frames"
+        extract_keyframes("fake.mp4", str(output_dir), interval_seconds=5)
+        cmd = mock_run.call_args[0][0]
+        assert "-q:v" in cmd and "2" in cmd
+
+    @patch("video_processor.subprocess.run")
+    def test_raises_on_ffmpeg_failure(self, mock_run, tmp_path):
+        mock_run.side_effect = subprocess.CalledProcessError(1, "ffmpeg")
+        output_dir = tmp_path / "frames"
+        with pytest.raises(subprocess.CalledProcessError):
+            extract_keyframes("bad.mp4", str(output_dir))
+
+
+# ─── generate_vertical_reel_clip ─────────────────────────────────────────────
+
+class TestGenerateVerticalReelClip:
+
+    @patch("video_processor.subprocess.run")
+    def test_blurred_mode_uses_boxblur_filter(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")          # must exist — validation guard checks os.path.exists
+        out = str(tmp_path / "reel.mp4")
+        generate_vertical_reel_clip(str(src), 0.0, 30.0, out, render_mode="blurred")
+        cmd = mock_run.call_args[0][0]
+        vf_value = cmd[cmd.index("-vf") + 1]
+        assert "boxblur" in vf_value
+        assert "split" in vf_value
+
+    @patch("video_processor.subprocess.run")
+    def test_center_mode_uses_center_crop(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        out = str(tmp_path / "reel.mp4")
+        generate_vertical_reel_clip(str(src), 0.0, 30.0, out, render_mode="center")
+        cmd = mock_run.call_args[0][0]
+        vf_value = cmd[cmd.index("-vf") + 1]
+        assert "(iw-ow)/2" in vf_value
+
+    @patch("video_processor.subprocess.run")
+    def test_left_mode_uses_zero_x_offset(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        out = str(tmp_path / "reel.mp4")
+        generate_vertical_reel_clip(str(src), 10.0, 40.0, out, render_mode="left")
+        cmd = mock_run.call_args[0][0]
+        vf_value = cmd[cmd.index("-vf") + 1]
+        assert "crop" in vf_value
+        assert ":0," in vf_value or vf_value.endswith(":0")
+
+    @patch("video_processor.subprocess.run")
+    def test_right_mode_uses_iw_ow_offset(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        out = str(tmp_path / "reel.mp4")
+        generate_vertical_reel_clip(str(src), 5.0, 25.0, out, render_mode="right")
+        cmd = mock_run.call_args[0][0]
+        vf_value = cmd[cmd.index("-vf") + 1]
+        assert "iw-ow" in vf_value
+
+    @patch("video_processor.subprocess.run")
+    def test_output_uses_libx264_and_aac(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        out = str(tmp_path / "reel.mp4")
+        generate_vertical_reel_clip(str(src), 0.0, 30.0, out)
+        cmd = mock_run.call_args[0][0]
+        assert "libx264" in cmd
+        assert "aac" in cmd
+
+    @patch("video_processor.subprocess.run")
+    def test_removes_existing_output_before_render(self, mock_run, tmp_path):
+        mock_run.return_value = MagicMock(returncode=0)
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        out = tmp_path / "reel.mp4"
+        out.write_bytes(b"old content")
+        generate_vertical_reel_clip(str(src), 0.0, 10.0, str(out))
+        assert not out.exists()
+
+    @patch("video_processor.subprocess.run")
+    def test_raises_on_ffmpeg_failure(self, mock_run, tmp_path):
+        mock_run.side_effect = subprocess.CalledProcessError(
+            1, "ffmpeg", stderr=b"encoding error"
+        )
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        with pytest.raises(subprocess.CalledProcessError):
+            generate_vertical_reel_clip(str(src), 0.0, 30.0, str(tmp_path / "out.mp4"))
+
+    def test_raises_when_end_before_start(self, tmp_path):
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        with pytest.raises(ValueError, match="end_time"):
+            generate_vertical_reel_clip(str(src), 30.0, 10.0, str(tmp_path / "out.mp4"))
+
+    def test_raises_when_negative_start(self, tmp_path):
+        src = tmp_path / "src.mp4"
+        src.write_bytes(b"fake")
+        with pytest.raises(ValueError, match="negative"):
+            generate_vertical_reel_clip(str(src), -5.0, 10.0, str(tmp_path / "out.mp4"))
+
+    def test_raises_when_source_missing(self, tmp_path):
+        with pytest.raises(FileNotFoundError):
+            generate_vertical_reel_clip("nonexistent.mp4", 0.0, 10.0, str(tmp_path / "out.mp4"))