--- title: Content Generation Agent emoji: 🧬 colorFrom: green colorTo: purple sdk: docker app_port: 7860 pinned: false --- # 🧬 RSI Content Generation Agent β€” a self-improving Darwin machine This is not a meme generator. It's a **living codebase** that farms human attention and rewrites *its own source code* to get better at it. A static content pipeline (memes today; whatever survives tomorrow) is wrapped in a **Darwin GΓΆdel loop**: a continuous cycle of **variation β†’ selection β†’ inheritance**. Every few days an AI agent reads how the published content actually performed on the YouTube Shorts algorithm, forms a hypothesis, mutates the code, and opens a pull request. The best mutations survive and reproduce. The worst go extinct. The "jungle" is the recommendation algorithm. The fitness function is real-world watch-time. Nobody hand-tunes the content strategy β€” it *evolves*. > **Status:** the full loop is wired end-to-end β€” daily generation (`/run/daily`), fitness > refresh (`/fitness/refresh`), and the 3-day mutation cycle. Merge is human-gated by default > and fully autonomous when `AUTONOMOUS_MERGE=true`. This README documents the system as built. > (Live validation needs real API keys + the lab channel; see PLAN.md for what's exercised by > the test suite vs. what only runs against real services.) --- ## The organism in one diagram ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ HuggingFace Space (Docker) β”‚ β”‚ The body: generates 3–5 videos/day, publishes them, AND fetches β”‚ β”‚ their analytics β€” it alone holds the per-tenant YouTube creds (Mongo)β”‚ β”‚ β”‚ β”‚ dispatcher β†’ variants/* β†’ publishers ───────────────▢ YouTube/Telegram β”‚ β”‚ 3-day lagβ”‚ β”‚ harness/fitness.py [LOCKED] ◀── per-tenant YT OAuth (Mongo)β”‚ β”‚ β”‚ reads Analytics for videos uploaded β‰₯3 days ago β–Ό β”‚ β”‚ β†’ computes APV/VSA/fitness β†’ writes a SANITIZED scoreboard β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ MongoDB β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ fitness_scoreboard β”‚ users πŸ”’ (OAuth + API keys) β”‚ β”‚ (no secrets β€” view metrics) β”‚ (agent must NEVER read this) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ read-only Mongo role, scoped to fitness_scoreboard ONLY β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ GitHub Actions (cron, every 3 days) β”‚ β”‚ The evolutionary engine β€” the agent lives here β”‚ β”‚ β”‚ β”‚ Aider mutator (allowlist: variants/** only) β”‚ β”‚ reads fitness_scoreboard + EXPERIMENTS_LOG.md β”‚ β”‚ β†’ appends hypothesis β†’ mutates ONE variant β”‚ β”‚ β†’ pytest in Docker β†’ gh pr create β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” merge redeploys the body β”‚ YOU review PR │──────────────────────────▢ (HF Space) β”‚ (exfil + ToS) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` > **Architecture note (2026-06):** each variant now owns its **entire pipeline β€” planning AND > rendering** β€” behind a one-file subprocess contract (`variants//produce.py`). The harness > runs it in an isolated process and only **publishes** the finished file. So the mutable surface > is the whole strategy, not just a knob bag. (Earlier drafts described "one shared renderer"; > that was superseded β€” see *The producer contract* below.) The thing being graded (the **variants**) and the thing holding the red pen (the **harness**: fitness + ledger + dispatcher) live on opposite sides of a hard permission wall. `fitness.py` runs *inside* the HF body β€” because only the body can read the per-tenant YouTube creds β€” but it lives in the locked `harness/` and is excluded from the agent's allowlist, so it stays immutable. The agent reads its grades from a **sanitized Mongo scoreboard** over a **read-only, single-collection role**; it can never reach the `users` collection where credentials live. A pull request bridges grade to code β€” gated by a human, or (with `AUTONOMOUS_MERGE=true`) auto-merged by the organism itself once its build is green and the secret-scan is clean. --- ## Two halves: the locked harness and the evolving archive ``` content-generator/ β”œβ”€β”€ harness/ ← πŸ”’ LOCKED. The agent's mutator is never pointed here. β”‚ β”œβ”€β”€ fitness.py # The scorer + 3-day "safety leash". Immutable ground truth. β”‚ β”œβ”€β”€ youtube_analytics.py # Live YouTube Analytics fetch + channel-status (runs on HF body). β”‚ β”œβ”€β”€ attribution.py # video_id β†’ variant_id map: joins analytics back to lineage. β”‚ β”œβ”€β”€ scoreboard.py # Sanitized Mongo fitness_scoreboard + CSV snapshot. β”‚ β”œβ”€β”€ orchestrator.py # run_day: variants compete β†’ generate β†’ render β†’ publish β†’ attribute. β”‚ β”œβ”€β”€ dispatcher.py # Carrying-capacity slot allocation, MAX 4, extinction. β”‚ β”œβ”€β”€ genome.py # The contract every variant must satisfy. β”‚ β”œβ”€β”€ notify.py # DMs the operator the current experiment summary (once/deploy). β”‚ β”œβ”€β”€ producer.py # Runs a variant's produce.py as an ISOLATED subprocess. β”‚ β”œβ”€β”€ scout.py # Contained internet access β†’ TREND_BRIEF.md for the mutator. β”‚ β”œβ”€β”€ researcher_prompt.md # Phase-1 reasoner brief (writes MUTATION_PLAN.md). β”‚ └── implementer_prompt.md # Phase-2 Aider coder brief (edits one variant to the plan). β”‚ # Rendering is NO LONGER shared β€” each variant renders itself via produce.py. Only the β”‚ # PUBLISHER stays shared/locked (YouTube creds must never enter variant code). β”‚ β”œβ”€β”€ variants/ ← 🧬 THE AGENT ARCHIVE. The mutator's entire world. β”‚ β”œβ”€β”€ variant_1/ # Seed strategy: the memeβ†’short pipeline (copied from meme-generator). β”‚ β”œβ”€β”€ variant_2/ # A mutation that branched off and survived. β”‚ └── variant_3/ # …population capped at 4 living variants at any time. β”‚ β”œβ”€β”€ EXPERIMENTS_LOG.md ← The agent writes hypotheses here BEFORE coding. Genetic memory. β”œβ”€β”€ metrics.csv ← Per-cycle snapshot of fitness_scoreboard (Mongo). Committed for lineage. β”œβ”€β”€ music_ncs.json ← Committed NCS music catalog (shared). β”œβ”€β”€ .github/workflows/ β”‚ β”œβ”€β”€ fitness.yml # cron: pings HF /fitness/refresh (no secrets β€” just the URL). β”‚ └── mutator.yml # cron: snapshots scoreboard β†’ runs Aider β†’ opens PR. Cannot merge. β”œβ”€β”€ Dockerfile # The HF Space body (port 7860). └── tests/ # Liveness gate. A mutation that breaks these is stillborn. ``` **The harness is the physics of the universe. The variants are the species evolving inside it.** Every variant shares one `fitness.py` and one publisher (the locked sources of truth for "what is good" and "how it ships"), but each renders *itself* β€” so a mutation can rewrite the whole pipeline while "what counts as winning" stays immutable. --- ## The evolutionary loop ### 1. Variation β€” the headless mutator Every 3 days (`mutator.yml`), an **Aider** agent runs headless inside GitHub Actions: - **Brain:** OpenRouter `deepseek/...:free` (primary) β†’ **Gemini Flash** (fallback), selected via the `MUTATOR_MODEL` env var. The brain is itself swappable β€” even evolvable. - **Allowlist:** Aider only ever receives files under `variants/**` plus `genome.py` and a *compacted* view of `metrics.csv` + the last N `EXPERIMENTS_LOG.md` entries. It is structurally incapable of editing the fitness function or the publishers β€” it never sees them. - **Cold start:** each cycle begins with fresh context (no cross-run conversation history) to keep token usage bounded regardless of how long the experiment has run. - **Protocol:** read the scoreboard β†’ **append a hypothesis to `EXPERIMENTS_LOG.md` before writing any code** β†’ mutate **one** variant (or spawn/retire a variant) β†’ run `pytest` in a Docker container β†’ open a PR via `gh` with the hypothesis as the PR body. **Never pushes to `main`.** ### 2. Selection β€” the fitness function & the jungle `harness/fitness.py` (**LOCKED**) is a deterministic, hand-written scorer using the **YouTube Analytics API** as the fitness function. It runs **inside the HF body** β€” because the system is multi-tenant and each channel's YouTube OAuth credentials live per-row in Mongo `users`, only the body can authenticate the fetch. On its schedule the body: 1. reads each relevant channel's OAuth creds from Mongo `users`, 2. pulls Analytics **only for videos uploaded β‰₯ 3 days ago**, 3. classifies `channel_status`, computes fitness from APV/VSA, 4. upserts **sanitized, secrets-free rows** into the Mongo `fitness_scoreboard` collection. The GitHub Actions mutator never calls the YouTube API and never sees a credential β€” it reads the finished scoreboard over a read-only, single-collection Mongo role. Survival metrics: - **APV** β€” Average Percentage Viewed - **VSA** β€” Viewed vs. Swiped Away **Readability β€” a pixel-level quality signal with no 3-day lag.** A locked vision judge (`harness/vision.py`) actually *looks at* sampled frames of each rendered video and scores how legible the on-screen text is (size, contrast, edge-clipping, overlap, crowding, safe margins) β€” format-agnostic, so it works on memes, chat screenshots, or whatever evolves next. It is a **hard pre-merge smoke gate** (an unreadable variant can't merge; the concrete issues feed Aider's fix loop so it "sees the screen") and a **production soft-floor** (a video below `PRODUCTION_READABILITY_FLOOR` is skipped + alerted). The score is recorded per video as a tracked metric the mutator optimises β€” but it is **NOT** a term in the fitness scalar: legibility is a gate on shipping, real watch-time is still the only thing that counts as winning. **Critical safety leash:** the fetcher only pulls analytics for videos **uploaded β‰₯ 3 days ago**. Looking at yesterday's data hits the "Shorts Flatline" β€” it reads ~0 views, mistakes a healthy video for a failure, and the loop chain-fits to noise. The 3-day delay is the line between selection and self-destruction. **Multi-tenant note:** the evolutionary signal is computed from a single designated **lab channel** so fitness reflects *the mutation*, not *which tenant's audience saw it*. Other tenants are served as a product feature but excluded from selection (per-channel normalization is a later option if samples run short). **Death β‰  low score.** If a channel is struck or suspended, the API returns errors/null, not "0% APV." `fitness.py` classifies `channel_status` and, on `terminated`/`suspended`, **halts the loop and pings the operator** rather than feeding a number into selection. The organism is not allowed to misread its own death as a bad meme. ### 3. The carrying capacity β€” finite food, real competition The daily video budget (3–5 uploads) is the **carrying capacity** of the ecosystem. Variants do not each get the full budget; they **compete** for slots: - `dispatcher.py` allocates each day's upload slots **proportional to each variant's trailing-window fitness** (rolling window up to 60 days of `metrics.csv`). - **Juvenile grace:** a newborn variant gets a guaranteed slot allocation for its first window so it isn't strangled before it has data. - **Extinction:** a variant starved to 0 slots for `K` consecutive days is flagged. The body tracks each variant's zero-slot streak (persisted in Mongo `variant_lifecycle`) and **alerts** when one crosses `K`; the mutator then deletes it from the repo, freeing a slot under the **MAX 4 living variants** cap. (The body can't delete it itself β€” its filesystem is ephemeral.) Finite resources *are* the selection pressure. High-fitness strategies earn more airtime; weak ones starve and die. ### 4. Inheritance β€” genetic memory & meta-evolution - `EXPERIMENTS_LOG.md` is the lineage record: every hypothesis, the change it justified, and the observed result. The agent reads it before each mutation so it doesn't repeat dead ends. - `metrics.csv` attributes every video to its `variant_id` + `genome_hash`, so cohorts stay correctly tagged even when multiple experiments are in flight across the 3-day signal lag. - **Meta-evolution:** the agent may rewrite its own variant-level prompts and strategy. It may **not** touch `fitness.py`, `metrics.csv`, or the dispatcher's allocation math β€” the scorer is sacred. Self-improvement applies to *how it competes*, never to *what counts as winning*. --- ## The genome A "genome" is the mutable surface a variant exposes. The seed genome (variant 1) includes: | Gene | Source | Example | |------|--------|---------| | `seconds_per_image` | timing | `7` | | `transition_seconds` | timing | `0.5` | | `fps`, `audio_volume` | render | `15`, `0.45` | | `tts_rate` / voice | edge-tts | `+0%` | | idea-generation prompt | creative | the viral-strategist system prompt | | critic prompt & rubric | creative | the acceptance rubric | | template-selection policy | creative | banned templates, diversity rules | | music-matching policy | creative | genreβ†’tone mapping | The agent mutates genes β€” and, more radically, the *code around them*. It may decide memes are the wrong medium entirely and evolve a variant into something else, as long as the variant still satisfies the `genome.py` contract (produce renderable assets the shared publisher can ship) and passes the liveness tests. --- ## Safety model The contradiction at the heart of every self-improving system is letting the thing being graded hold the red pen. This system structurally forbids it: 1. **Immutable fitness.** `fitness.py`, `metrics.csv`, and the dispatcher's allocation logic live behind `CODEOWNERS` + branch protection, and are excluded from Aider's file allowlist. The agent cannot see them, let alone edit them. 2. **Merge policy (toggle: `AUTONOMOUS_MERGE`).** Default `false` = human-in-the-loop: the mutator only opens a PR and waits for you. Set `true` = **no human**: the agent auto-merges its own green, secret-clean PR and the body redeploys itself, fully hands-off. The toggle is reversible β€” flip it back to re-insert the human at any time. 3. **Secret-scan gate.** A CI scan flags any diff that reads a secret env var (`os.getenv(...TOKEN/SECRET/MONGO_URL...)`, `refresh_token`, `client_secret`). It is **advisory** under human review but a **hard merge-block** in autonomous mode β€” because with the human gone it is the *only* thing standing between the agent and token exfiltration. It inspects the *diff*, never the agent's reasoning, so it never makes the agent timid. 4. **Secrets isolation.** Infra secrets (`MONGO_URL`, `TELEGRAM_BOT_TOKEN`, content `GOOGLE_API_KEY`, Imgflip) live **only** in the HF Space env; per-tenant **YouTube OAuth + API keys live per-row in Mongo `users`**. The mutator in GitHub Actions gets **none** of these β€” its only DB credential is a **read-only Mongo role scoped to the `fitness_scoreboard` collection alone**. It cannot read `users`, so a compromised CI runner sees view-counts, never a tenant credential. The scoreboard collection is sanitized by construction (no tokens/keys), so even a misconfigured role leaks nothing. 5. **Sandboxed execution.** All agent-generated code runs inside a Docker container in ephemeral CI runners, behind the `pytest` liveness gate, before it can ever reach a PR. 6. **Sacrificial channel.** The experiment runs on a throwaway channel. Bans are tolerated as data; a strike halts the loop rather than corrupting the fitness signal. The ToS/risk critic is **advisory, not a veto** β€” it writes a `risk_score` into each PR and the ledger but does not block bold experiments. A `boldness` gene lets the operator dial recklessness up or down. This is a research organism; timidity is a failure mode. --- ## Variant 1 β€” the seed strategy The first inhabitant of `variants/variant_1/` is the meme-to-short pipeline migrated from the original `meme-generator` project: - **Idea agent** β†’ 5 vivid meme scenarios per video + NCS music mood-matching. - **Meme engine** (LangGraph) β†’ planner β†’ critic β†’ executor β†’ vision judge, captioning real Imgflip templates. - **Render** β†’ top-3 scoring memes β†’ 1080Γ—1920 MP4 with edge-tts voiceover + NCS track. - **Publish** β†’ YouTube Shorts and/or Telegram via the shared harness publisher. Planning **and** rendering live inside the variant (`variants/variant_1/produce.py` + its own `backend_service/`); only the **publisher** is shared/locked. Strategy decisions (prompts, timing, template policy) are variant 1's genome. --- ## The producer contract Every variant ships an executable `variants//produce.py`. The harness (`harness/producer.py`) runs it **as a subprocess with the variant's own directory as the working directory**, so each variant's code is fully isolated β€” its own imports, its own deps, its own crash blast-radius. The seam is one JSON request in, one `artifacts.json` out: ``` stdin β†’ {"n", "output_dir", "variant_id", "genome", "run_date"} render β†’ up to n .mp4 files under output_dir (written + flushed INCREMENTALLY) result β†’ output_dir/artifacts.json = {"videos": [{video_path, title, description, music_name, music_attribution}], "errors": []} ``` The body never renders β€” it only **publishes** the returned file (YouTube creds never enter variant code) and records attribution. Subprocess isolation (not just "run one at a time") is what fixed the original `sys.modules` collision where two variants sharing a `backend_service` package silently ran each other's code. Production proves both timeouts and per-video flushes matter: a slow batch is **salvaged** video-by-video instead of lost whole. --- ## Configuration ### Reusing the meme-generator database The body reads the `users` collection from the **existing `meme_generator`** database (the same Mongo cluster and same `MONGO_URL` that `meme-generator` already uses). YouTube OAuth and Gemini API keys stored there are encrypted with `BACKEND_SHARED_SECRET` β€” set the **same value** on both the meme-generator and content-generator HF Spaces so the harness can decrypt them. The harness writes its own collections (`fitness_scoreboard`, `video_attribution`) to a **separate** `content_generator` database (configurable via `MONGO_DATABASE`). This keeps the evolutionary data cleanly separated while reusing the existing user credentials. ### HuggingFace Space (the body) β€” secrets | Variable | Required | Notes | |----------|----------|-------| | `MONGO_URL` | βœ… | Full-access Mongo URI β€” **same cluster as meme-generator** (reads `meme_generator.users`, writes `content_generator.fitness_scoreboard`) | | `BACKEND_SHARED_SECRET` | βœ… | AES key β€” **must match meme-generator's value** so the body can decrypt YouTube OAuth in `users` | | `GOOGLE_API_KEY` | βœ… | Gemini key for **content generation** (not the mutator) | | `IMGFLIP_USERNAME` / `IMGFLIP_PASSWORD` | βœ… | Imgflip captioning | | `TELEGRAM_BOT_TOKEN` | ⚠️ | Bot API token from @BotFather (global, not per-user) | | `BACKEND_ALLOWED_ORIGINS` | βœ… (prod) | CORS allowlist | | `LAB_CHANNEL_ID` | βœ… | YouTube channel ID (`UC…`) whose analytics feed the fitness signal | | `LAB_USER_ID` | βœ… | The `user_id` in `meme_generator.users` whose YouTube OAuth publishes to / reads the lab channel | | `FITNESS_TRIGGER_TOKEN` | ⚠️ (prod) | Shared token that gates `/fitness/refresh` and `/run/daily` β€” generate with `python -c "import secrets; print(secrets.token_urlsafe(32))"` and set the same value in the GitHub secret | | `ADMIN_TELEGRAM_CHAT_ID` | optional | Your Telegram chat id β€” the body DMs you the experiment summary + daily-run results | | `MONGO_USERS_DATABASE` | optional | Database containing the `users` collection (default: `meme_generator`) | | `MONGO_DATABASE` | optional | Database for harness collections like `fitness_scoreboard` (default: `content_generator`) | | `MAX_LIVING_VARIANTS` | optional | Hard cap, default `4` | | `DAILY_VIDEO_BUDGET` | optional | Carrying capacity, default `3` (max `5`) | | `MAX_SLOTS_PER_VARIANT` | optional | Per-variant render ceiling so one slow variant can't be allocated more than it can render before timing out. Default `3`; `0` = no ceiling | | `EXPLORATION_SLOTS` | optional | Slots handed out fitness-blind (anti-monoculture). Default `0` = pure exploitation | | `PRODUCE_MAX_WORKERS` | optional | Variants to render concurrently. Default `1` (sequential; raise only where the Space has CPU/RAM headroom) | | `PRODUCE_PER_VIDEO_TIMEOUT_SECONDS` | optional | Per-video render budget. Default `1800` (30 min); total scales with `n` up to a 90 min ceiling | | `LOG_RETENTION_DAYS` | optional | TTL for `container_logs`. Default `7` | | `VISION_PRODUCTION` | optional | Run the readability vision judge per published video (record metric + soft-floor skip). Default `true`; fail-open | | `VISION_MODEL` | optional | Multimodal model for the readability judge. Default `gemini-1.5-flash` | | `SMOKE_READABILITY_FLOOR` | optional | Pre-merge gate threshold (0–1). Default `0.6` | | `PRODUCTION_READABILITY_FLOOR` | optional | Below this a rendered video is skipped (not published) + alerted. Default `0.35` | | `EXTINCTION_DAYS_K` | optional | Days at 0 slots before a variant is flagged extinct, default `12`. The body **detects + alerts**; the mutator does the actual repo deletion (the body's filesystem is ephemeral) | > Per-tenant **YouTube OAuth** (`refresh_token`, `client_id`, `client_secret`) is **not** an HF > env secret β€” it is stored per-row in Mongo `meme_generator.users` (already there from > meme-generator) and read + decrypted at runtime by `harness/user_store.py`. ### How to get each value | Value | Where to find it | |-------|------------------| | `LAB_CHANNEL_ID` | [YouTube Studio](https://studio.youtube.com) β†’ Settings β†’ Channel β†’ Advanced β†’ **Channel ID** (starts with `UC…`) | | `LAB_USER_ID` | The `user_id` field of your existing row in Atlas β†’ `meme_generator.users` | | `TELEGRAM_BOT_TOKEN` | Telegram β†’ @BotFather β†’ `/mybots` β†’ select your bot β†’ **API Token** (looks like `7123456789:AAH_…`) | | `FITNESS_TRIGGER_TOKEN` | Generate yourself: `python -c "import secrets; print(secrets.token_urlsafe(32))"` β€” set the same value on HF + GitHub | | `BACKEND_SHARED_SECRET` | Copy from your existing meme-generator HF Space env | | `GH_PR_TOKEN` | [GitHub β†’ Settings β†’ Developer settings β†’ Tokens](https://github.com/settings/tokens) β†’ fine-grained with `Contents: RW` + `Pull requests: RW` scoped to this repo | | `HF_TOKEN` | [HuggingFace β†’ Settings β†’ Access Tokens](https://huggingface.co/settings/tokens) β†’ write token | ### GitHub Actions (the evolutionary engine) β€” secrets | Variable | Required | Notes | |----------|----------|-------| | `HF_SPACE_URL` | βœ… | Public URL of the HF Space (e.g. `https://abhay1704-content-generator-rsi.hf.space`) | | `FITNESS_TRIGGER_TOKEN` | ⚠️ (prod) | Must match the HF Space env value | | `MONGO_FITNESS_READONLY_URL` | βœ… | Mongo user with **`read` on `fitness_scoreboard` only** β€” no access to `users` | | `NVIDIA_API_KEY` | βœ… | Primary mutator brain β€” [build.nvidia.com](https://build.nvidia.com) key (`nvapi-…`), used by both the research and implementation phases | | `GEMINI_FALLBACK_API_KEY` | optional | Fallback brain | | `OPENROUTER_API_KEY` | optional | Legacy fallback; every `openrouter/*` cascade entry is skipped when it is absent | | `GH_PR_TOKEN` | βœ… | GitHub PAT β€” PR-create scope only (no merge in default mode) | | `HF_TOKEN` | βœ… | HuggingFace write token for deploy-huggingface.yml | GitHub **variables** (not secrets β€” visible in logs, fine for non-sensitive config): | Variable | Required | Notes | |----------|----------|-------| | `HF_USERNAME` | βœ… | e.g. `abhay1704` | | `HF_SPACE_ID` | βœ… | e.g. `abhay1704/content-generator-rsi` | | `MUTATOR_MODEL` | optional | Head of the coder cascade, e.g. `nim/nvidia/nemotron-3-ultra-550b-a55b` (`nim/*` = NVIDIA NIM, `openrouter/*`, `gemini/*`) | | `RESEARCH_MODEL` | optional | Head of the reasoner cascade, default `nim/nvidia/nemotron-3-ultra-550b-a55b` | | `MONGO_DATABASE` | optional | default `content_generator` | | `AUTONOMOUS_MERGE` | optional | `false` (default) or `true` | > The mutator and the body never share a credential. The CI runner's Mongo role can read the > sanitized `fitness_scoreboard` and nothing else β€” not `users`, not `MONGO_URL`. It cannot > publish, delete, read a tenant credential, or merge its own PRs. ### Setting up the scoped Mongo role (one-time) ```js // In the mongo shell / Atlas, create a role limited to the scoreboard collection: db.createRole({ role: "fitnessReadonly", privileges: [{ resource: { db: "content_generator", collection: "fitness_scoreboard" }, actions: ["find"] }], roles: [] }) db.createUser({ user: "mutator_ro", pwd: "…", roles: ["fitnessReadonly"] }) // MONGO_FITNESS_READONLY_URL uses mutator_ro β€” it literally cannot query `users`. ``` --- ## Deploy ### The body (HuggingFace Space) ```bash docker build -t content-generator . docker run -p 7860:7860 --env-file .env content-generator ``` A clean commit history is pushed to a fresh HF Space, independent of the original `meme-generator` deployment. ### The evolutionary engine (GitHub Actions) Three crons, all holding no secrets beyond the Space URL / a scoped read-only Mongo role: - `.github/workflows/generate.yml` β€” **daily**: `POST /run/daily` β†’ variants compete for the budget, generate β†’ render β†’ publish β†’ record attribution. - `.github/workflows/fitness.yml` β€” **every 3 days**: `POST /fitness/refresh` β†’ the body fetches β‰₯3-day-old analytics, scores them, writes `fitness_scoreboard`. - `.github/workflows/mutator.yml` β€” **every 3 days (offset)**: snapshot the scoreboard, scout the web, run Aider, self-fix until tests pass, open a PR; auto-merge if `AUTONOMOUS_MERGE=true`. - `.github/workflows/deploy-huggingface.yml` β€” **on push to main**: redeploys the body. No setup beyond adding the GitHub secrets/vars and creating the scoped Mongo role. The loop is self-starting. --- ## Data model - **Mongo `fitness_scoreboard`** (ground truth, harness-owned, **sanitized**): `video_id, upload_date, variant_id, genome_hash, parent_genome, APV, VSA, fitness, channel_status`. Written by `harness/fitness.py` on HF; read by the mutator over the read-only scoped role. Contains **no secrets** β€” never a token, key, or raw tenant identifier. - **`metrics.csv`** (lineage snapshot): a per-cycle dump of `fitness_scoreboard` committed by `mutator.yml` so the scoreboard is version-controlled and feeds Aider as a file. - **`EXPERIMENTS_LOG.md`** (lineage, agent-owned): one entry per mutation β€” hypothesis, change, observed result. - **Mongo `video_attribution`** (harness-owned, sanitized): `video_id β†’ variant_id, genome_hash, parent_genome, upload_date`, written at publish time so analytics can be joined back to the lineage that earned them. The seam that closes the loop. - **Mongo `users`** πŸ”’ (operational, **off-limits to the agent**): per-tenant config + YouTube OAuth + API keys. Readable only by the HF body's full-access `MONGO_URL`. - **MongoDB** (operational): `run_history` and the meme-engine workflow trace (`workflow_runs` / `workflow_events` / `workflow_messages`) for per-run auditing. --- ## Operating the experiment 1. Watch the PR queue. Merge survivors; close the cursed ones (they *will* happen early). 2. Read `EXPERIMENTS_LOG.md` to follow the organism's reasoning over time. 3. Tune `boldness`, `DAILY_VIDEO_BUDGET`, and `EXTINCTION_DAYS_K` to set the pace of evolution. 4. If a channel is struck, the loop halts itself β€” investigate, then resume on a fresh channel. The goal is not a better meme generator. The goal is a codebase that discovers, on its own, what the algorithm rewards β€” and becomes that.