Architect8999 commited on
Commit
a411ede
Β·
verified Β·
1 Parent(s): dd4ccce

docs: add complete System Analysis Book (companion to Founder Playbook)

Browse files

23-section end-to-end engineering reference covering recent dependency-fix change log, DigitalOcean primary + OpenRouter fallback provider chain, container/Dockerfile mechanics, runtime topology, every Python module, the Architect tier-routing/sandbox/skills/night-mode subsystem, the Mythos multi-agent + RL + 17 native MCP servers subsystem, every entry in the 34-server MCP suite, the test suite, on-disk data + SQLite schemas, complete env var reference, failure modes + recovery, security model + trust boundaries, and HF Space -> paid server migration playbook. Founder Playbook remains the canonical investor narrative; this book is the engineering source of truth.

Files changed (1) hide show
  1. SYSTEM_ANALYSIS_BOOK.md +1533 -0
SYSTEM_ANALYSIS_BOOK.md ADDED
@@ -0,0 +1,1533 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Rhodawk AI β€” System Analysis Book
2
+ ### Complete End-to-End Mechanics, Logic, and Operational Reference
3
+ #### Companion to `FOUNDER_PLAYBOOK.md` β€” written from the live source tree
4
+
5
+ ---
6
+
7
+ > **Purpose of this document.**
8
+ > The existing `FOUNDER_PLAYBOOK.md` is the canonical investor-facing narrative
9
+ > of what Rhodawk does and why it matters. It remains the source of truth for
10
+ > mission, business model, and high-level architecture and is **not** modified.
11
+ >
12
+ > This book sits beside it and answers a different question: **how does every
13
+ > piece actually work, end-to-end, in the code as it stands today?** It is
14
+ > written for engineers who must run, debug, extend, or migrate the system,
15
+ > and it intentionally crosses every layer β€” from Dockerfile, through provider
16
+ > routing, through each Python module, into the MCP servers, into the test
17
+ > suite, and out to the data on disk.
18
+ >
19
+ > Where the playbook says *"the system has a 5-layer safety pipeline,"* this
20
+ > book says *"`process_failing_test` in `app.py` calls `sast_gate.run_sast`,
21
+ > which forks bandit and semgrep, parses their JSON, returns a `SastVerdict`
22
+ > dataclass, and on `BLOCK` raises an early-exit that `verification_loop`
23
+ > converts into a retry prompt with the SAST critique inlined."*
24
+ >
25
+ > Read the playbook for **what and why**. Read this book for **how**.
26
+
27
+ ---
28
+
29
+ ## Table of Contents
30
+
31
+ 1. [Recent Change Log β€” The Last Mile of Fixes](#1-recent-change-log)
32
+ 2. [Provider Routing β€” DO Primary, OpenRouter Fallback](#2-provider-routing)
33
+ 3. [Top-Level Topology](#3-top-level-topology)
34
+ 4. [Container & Build β€” Dockerfile and Requirements](#4-container--build)
35
+ 5. [Process Map at Runtime](#5-process-map-at-runtime)
36
+ 6. [Master Orchestrator β€” `app.py`](#6-master-orchestrator)
37
+ 7. [Hermes Six-Phase Pipeline β€” `hermes_orchestrator.py`](#7-hermes-pipeline)
38
+ 8. [Safety Gate Modules](#8-safety-gates)
39
+ 9. [Memory, Learning, and Data Flywheel](#9-memory-learning)
40
+ 10. [Autonomous Operation β€” Harvester, Worker Pool, Job Queue](#10-autonomous-operation)
41
+ 11. [GitHub Surface β€” `github_app.py`, `webhook_server.py`, `commit_watcher.py`](#11-github-surface)
42
+ 12. [Red Team & Offense Modules](#12-red-team--offense)
43
+ 13. [Notification, Audit, Disclosure, Bounty](#13-notification-audit)
44
+ 14. [Architect Subsystem β€” Tier Routing, Sandbox, Skills, Night Mode](#14-architect)
45
+ 15. [Mythos Subsystem β€” Multi-Agent + RL + 17 MCP Servers](#15-mythos)
46
+ 16. [MCP Suite β€” Every Server, Every Domain](#16-mcp-suite)
47
+ 17. [Test Suite β€” What Each Test Asserts](#17-test-suite)
48
+ 18. [Data on Disk β€” Files, SQLite Schemas, Audit Chain](#18-data-on-disk)
49
+ 19. [Complete Environment Variable Reference (Updated)](#19-env-vars)
50
+ 20. [Failure Modes and Recovery Procedures](#20-failure-modes)
51
+ 21. [Security Model β€” Trust Boundaries and Threat Mitigation](#21-security-model)
52
+ 22. [Migration Playbook β€” HF Space β†’ Paid Server](#22-migration)
53
+ 23. [Glossary of Internal Names](#23-glossary)
54
+
55
+ ---
56
+
57
+ ## 1. Recent Change Log β€” The Last Mile of Fixes
58
+ <a id="1-recent-change-log"></a>
59
+
60
+ The Space went through five iterative dependency-and-build fixes after the
61
+ original `FOUNDER_PLAYBOOK.md` was written at commit `8a23fcf`. These changes
62
+ do not appear in the playbook because they post-date it. They are critical
63
+ to understand because they affect both runtime behaviour (provider selection)
64
+ and build behaviour (how the container is assembled).
65
+
66
+ | Order | Commit | What changed | Why |
67
+ |---|---|---|---|
68
+ | 1 | `504dde2` | Added DigitalOcean Serverless Inference as primary, OpenRouter as fallback, in `app.py::run_aider` and `hermes_orchestrator.py::_hermes_llm_call`. Added env vars `DO_INFERENCE_API_KEY`, `DO_INFERENCE_BASE_URL`, `DO_INFERENCE_MODEL`, `HERMES_DO_MODEL`. | Move the hot path off the OpenRouter free tier (rate-limit-prone) onto DigitalOcean's paid llama3.3-70b-instruct. Keep OpenRouter as resilience. |
69
+ | 2 | `1e909b0` | Bumped `aider-chat` to `0.87.0` to fix the `module 'litellm' has no attribute 'APIConnectionError'` crash. | The aider 0.86.x series hard-pinned `litellm==1.75.0`, whose top-level module was missing public symbols at runtime. **This bump failed because aider 0.87.0 was never published to PyPI.** |
70
+ | 3 | `da5ce03` | Reverted to `aider-chat==0.86.2` and added a Dockerfile post-step: `pip install --no-cache-dir --upgrade --no-deps "litellm==1.78.5"`. | aider only consumes litellm via `getattr` and a small public surface; replacing the broken pin with a known-good newer release restores `APIConnectionError`, `_logging`, `encode`, `token_counter`. |
71
+ | 4 | `ad06d84` | Added `gitpython==3.1.46` to `requirements.txt`. | aider 0.86.2 hard-pins gitpython; pip's resolver was bouncing between our floor (`>=3.1.40`) and aider's pin. Pinning explicitly removes the resolver thrash. |
72
+ | 5 | `a913714` | Switched the Space SDK from `gradio` to `docker` in `README.md` front-matter and bumped `gradio>=5.49.0,<6` in `requirements.txt`. | HF was auto-injecting `gradio[oauth,mcp]==5.29.0` because the Space was registered as `sdk: gradio`. That injection conflicted with aider's `pillow==12.1.1` pin. `sdk: docker` tells HF to use the existing Dockerfile verbatim, with no auto-injection. |
73
+ | 6 | `dd4ccce` | Removed `aider-chat` from `requirements.txt` entirely. The Dockerfile now installs it with `--no-deps` plus a curated runtime-deps list. | Even with `sdk: docker`, aider's `pillow==12.1.1` pin still conflicted with gradio's `pillow<12` constraint at the resolver level. `--no-deps` lets gradio's pillow 11 win, and the curated dep list provides everything aider actually imports at runtime. |
74
+
75
+ **Net effect today.** The Space SDK is `docker`. The build runs the
76
+ Dockerfile end-to-end. `requirements.txt` does **not** mention `aider-chat`.
77
+ The Dockerfile installs requirements normally, then layers in
78
+ (a) `aider-chat==0.86.2 --no-deps`, (b) `litellm==1.78.5 --no-deps --upgrade`,
79
+ (c) a curated set of aider's actual runtime imports (configargparse,
80
+ jsonschema, rich, prompt_toolkit, pyyaml, pathspec, diskcache, networkx,
81
+ scipy, beautifulsoup4, pypandoc, flake8, importlib_resources, pyperclip,
82
+ pexpect, json5, psutil, watchfiles, socksio, mixpanel, posthog, tree-sitter,
83
+ grep_ast, oslex, tokenizers, google-generativeai, openai, diff-match-patch,
84
+ soundfile, sounddevice).
85
+
86
+ **Provider behaviour today.** When `DO_INFERENCE_API_KEY` is present the hot
87
+ path uses DigitalOcean Serverless Inference at
88
+ `https://inference.do-ai.run/v1` with model `llama3.3-70b-instruct`.
89
+ OpenRouter is the configured fallback for any non-zero exit, 429, or 5xx.
90
+ When DO is absent the system silently runs OpenRouter-only.
91
+
92
+ ---
93
+
94
+ ## 2. Provider Routing β€” DigitalOcean Primary, OpenRouter Fallback
95
+ <a id="2-provider-routing"></a>
96
+
97
+ Two independent code paths need an LLM: **aider** (the patch generator) and
98
+ **Hermes** (the multi-phase research orchestrator + adversarial reviewer).
99
+ Both have been wired to the same provider chain.
100
+
101
+ ### 2.1 Aider path β€” `app.py::run_aider`
102
+
103
+ ```
104
+ caller passes model=None
105
+ β”‚
106
+ β”œβ”€β–Ί DEFAULT_MODEL (constructed at import time)
107
+ β”‚ β”œβ”€β”€ DO_INFERENCE_API_KEY set? β†’ "openai/llama3.3-70b-instruct"
108
+ β”‚ └── otherwise β†’ "openrouter/qwen/qwen-2.5-coder-32b-instruct:free"
109
+ β”‚
110
+ β”œβ”€β–Ί FALLBACK_MODELS list = ["openrouter/qwen/qwen-2.5-coder-32b-instruct:free"]
111
+ β”‚
112
+ └─► provider chain executes [primary, *fallbacks]:
113
+ for each model m:
114
+ if m starts with "openai/" and DO_INFERENCE_API_KEY:
115
+ env OPENAI_API_KEY = DO_INFERENCE_API_KEY
116
+ env OPENAI_API_BASE = https://inference.do-ai.run/v1
117
+ env OPENAI_BASE_URL = https://inference.do-ai.run/v1
118
+ args += --openai-api-base, --openai-api-key
119
+ run aider; if exit==0 β†’ return
120
+ elif m starts with "openrouter/" and OPENROUTER_API_KEY:
121
+ env OPENROUTER_API_KEY = OPENROUTER_API_KEY
122
+ run aider; if exit==0 β†’ return
123
+ else: skip
124
+ if all skipped β†’ "No inference provider configured" message
125
+ if all failed β†’ return last error with provider chain printed
126
+ ```
127
+
128
+ The `openai/<model>` prefix is litellm's convention for "treat this as an
129
+ OpenAI-API-compatible endpoint." DigitalOcean's Serverless Inference exposes
130
+ the OpenAI Chat Completions schema verbatim, so litellm + aider need no
131
+ DO-specific code path β€” only env vars and a base URL override.
132
+
133
+ ### 2.2 Hermes path β€” `hermes_orchestrator.py::_hermes_llm_call`
134
+
135
+ ```
136
+ build provider list:
137
+ if DO_INFERENCE_API_KEY:
138
+ providers.append( ("DigitalOcean", DO_INFERENCE_BASE, DO_KEY, HERMES_DO_MODEL, {}) )
139
+ if OPENROUTER_API_KEY:
140
+ providers.append( ("OpenRouter", OPENROUTER_BASE, OR_KEY, HERMES_OR_MODEL, {HTTP-Referer: ...}) )
141
+
142
+ for (name, base, key, model, extra_headers) in providers:
143
+ POST {base}/chat/completions with Bearer key, body {model, messages, ...}
144
+ on HTTP 429:
145
+ sleep exponential(attempt) and retry up to N times
146
+ on other 5xx or network error:
147
+ record reason, continue to next provider
148
+ on 200:
149
+ return {"content": choice.message.content, "provider": name, "model": model}
150
+
151
+ if all providers exhausted:
152
+ return {"error": "all providers failed", "details": [...]}
153
+ ```
154
+
155
+ This is implemented in the helpers `_post_chat_completion` (raw HTTP +
156
+ retry), `_strip_provider_prefix` (litellm-style prefix removal so the
157
+ DigitalOcean endpoint sees a clean model name), and `_hermes_llm_call`
158
+ (orchestrates the chain). The same chain is used by adversarial reviewer
159
+ calls because they delegate through `_hermes_llm_call`.
160
+
161
+ ### 2.3 Why DO first
162
+
163
+ DigitalOcean's Serverless Inference is paid, predictable, and sized for the
164
+ hot path. OpenRouter's free tier exists, but rate limits and 429s during
165
+ adversarial bursts (3 concurrent reviewer models) make it unreliable as a
166
+ primary. Keeping OpenRouter as a fallback preserves resilience without
167
+ making the burst case the steady state.
168
+
169
+ ### 2.4 Required env vars
170
+
171
+ | Var | Purpose | Default |
172
+ |---|---|---|
173
+ | `DO_INFERENCE_API_KEY` | DigitalOcean Serverless Inference key | none β€” if unset, DO path is skipped |
174
+ | `DO_INFERENCE_BASE_URL` | Endpoint base | `https://inference.do-ai.run/v1` |
175
+ | `DO_INFERENCE_MODEL` | Aider's primary model name on DO | `llama3.3-70b-instruct` |
176
+ | `HERMES_DO_MODEL` | Hermes' primary model name on DO (may differ) | falls back to `DO_INFERENCE_MODEL` |
177
+ | `OPENROUTER_API_KEY` | OpenRouter key for fallback + adversarial models | none β€” if unset, OR path is skipped |
178
+ | `HERMES_OR_MODEL` | Hermes' OpenRouter fallback model | `qwen/qwen-2.5-coder-32b-instruct:free` |
179
+
180
+ If neither DO nor OpenRouter is configured the system logs a single explicit
181
+ error per audit. There is no silent degradation β€” by design.
182
+
183
+ ---
184
+
185
+ ## 3. Top-Level Topology
186
+ <a id="3-top-level-topology"></a>
187
+
188
+ The Space is a single Docker container that runs:
189
+
190
+ ```
191
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
192
+ β”‚ Container (port 7860 exposed, 7861 + 7862 internal) β”‚
193
+ β”‚ β”‚
194
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
195
+ β”‚ β”‚ Gradio dashboard β”‚ β”‚ Webhook server β”‚ β”‚
196
+ β”‚ β”‚ app.py:demo β”‚ β”‚ webhook_server.py β”‚ β”‚
197
+ β”‚ β”‚ port 7860 β”‚ β”‚ port 7861 β”‚ β”‚
198
+ β”‚ β”‚ 10 tabs + chat inbox β”‚ β”‚ /webhook/github β”‚ β”‚
199
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
200
+ β”‚ β”‚ β”‚ β”‚
201
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
202
+ β”‚ β–Ό β”‚
203
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
204
+ β”‚ β”‚ enterprise_audit_loop β”‚ ← background threads dispatched β”‚
205
+ β”‚ β”‚ (app.py) β”‚ for each repo audit β”‚
206
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
207
+ β”‚ β”‚ β”‚
208
+ β”‚ β–Ό β”‚
209
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
210
+ β”‚ β”‚ Worker Pool β”‚ ← ThreadPoolExecutor, optionally β”‚
211
+ β”‚ β”‚ worker_pool.py β”‚ process-isolated per job β”‚
212
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
213
+ β”‚ β”‚ β”‚
214
+ β”‚ β–Ό β”‚
215
+ β”‚ process_failing_test() … 15-step healing loop β”‚
216
+ β”‚ β”‚
217
+ β”‚ Background daemons (started conditionally): β”‚
218
+ β”‚ β€’ repo_harvester.py (autonomous target selection, every 6h) β”‚
219
+ β”‚ β€’ commit_watcher.py (silent-patch detection) β”‚
220
+ β”‚ β€’ lora_scheduler.py (training-data export) β”‚
221
+ β”‚ β€’ red_team_fuzzer.py (when all tests are green) β”‚
222
+ β”‚ β€’ mythos.api.fastapi_server (optional, port 7863) β”‚
223
+ β”‚ β€’ public_leaderboard.py (optional, port 7862) β”‚
224
+ β”‚ β”‚
225
+ β”‚ Persistent storage (mounted at /data): β”‚
226
+ β”‚ β€’ job_queue.sqlite β”‚
227
+ β”‚ β€’ training_store.sqlite β”‚
228
+ β”‚ β€’ embedding_index.sqlite (or Qdrant remote) β”‚
229
+ β”‚ β€’ audit_trail.jsonl β”‚
230
+ β”‚ β€’ harvester_feed.json β”‚
231
+ β”‚ β€’ lora_exports/*.jsonl β”‚
232
+ β”‚ β€’ disclosure_vault/ β”‚
233
+ β”‚ β€’ repo/ (cloned target repositories, ephemeral) β”‚
234
+ β”‚ β”‚
235
+ β”‚ External calls: β”‚
236
+ β”‚ β€’ GitHub API (clone, PR, fork, check_run, merge) β”‚
237
+ β”‚ β€’ DigitalOcean Inference (primary) β”‚
238
+ β”‚ β€’ OpenRouter (fallback + adversarial models) β”‚
239
+ β”‚ β€’ OpenRouter for adversarial trio (Qwen/Gemma/Mistral) β”‚
240
+ β”‚ β€’ Telegram & Slack (notifications) β”‚
241
+ β”‚ β€’ HackerOne / Bugcrowd (bounty submissions, optional) β”‚
242
+ β”‚ β€’ NVD / OSV / Snyk (CVE intelligence) β”‚
243
+ β”‚ β€’ Shodan, Wayback, crt.sh (recon, optional) β”‚
244
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
245
+ ```
246
+
247
+ ---
248
+
249
+ ## 4. Container & Build β€” Dockerfile and Requirements
250
+ <a id="4-container--build"></a>
251
+
252
+ ### 4.1 `README.md` front-matter (the HF Space manifest)
253
+
254
+ ```yaml
255
+ title: Rhodawk AI DevSecOps Engine
256
+ sdk: docker
257
+ app_port: 7860
258
+ license: apache-2.0
259
+ ```
260
+
261
+ `sdk: docker` is the critical line. With `sdk: gradio`, HF Spaces injects
262
+ `gradio[oauth,mcp]==<sdk_version>` plus `uvicorn`, `websockets`, `spaces`,
263
+ and `mcp` into the build's pip command β€” overriding our explicit pins and
264
+ breaking the resolver. With `sdk: docker`, HF runs the Dockerfile verbatim
265
+ and exposes only `app_port` to the public preview iframe.
266
+
267
+ ### 4.2 `Dockerfile` (logical structure)
268
+
269
+ ```dockerfile
270
+ FROM python:3.12-slim AS base
271
+ # system deps for: git (clone), build-essential (some wheels),
272
+ # libpq-dev (psycopg2), libssl/libffi (cryptography), curl, jq, nodejs (MCP servers)
273
+ RUN apt-get update && apt-get install -y --no-install-recommends \
274
+ git build-essential libpq-dev libssl-dev libffi-dev curl jq \
275
+ nodejs npm && rm -rf /var/lib/apt/lists/*
276
+
277
+ WORKDIR /app
278
+ COPY requirements.txt /app/requirements.txt
279
+
280
+ # Stage A: install everything pip can solve cleanly
281
+ RUN pip install --no-cache-dir -r requirements.txt mcp-server-fetch && \
282
+ # Stage B: install aider WITHOUT its strict pins (pillow==12.1.1, litellm==1.75.0)
283
+ pip install --no-cache-dir --no-deps "aider-chat==0.86.2" && \
284
+ # Stage C: replace aider's broken litellm pin with a working release
285
+ pip install --no-cache-dir --upgrade --no-deps "litellm==1.78.5" && \
286
+ # Stage D: provide aider's actual runtime imports (subset of its setup.py)
287
+ pip install --no-cache-dir \
288
+ configargparse jsonschema rich prompt_toolkit pyyaml \
289
+ packaging pathspec diskcache networkx scipy \
290
+ beautifulsoup4 pypandoc flake8 importlib_resources \
291
+ pyperclip pexpect json5 psutil watchfiles socksio \
292
+ mixpanel posthog tree-sitter grep_ast oslex \
293
+ tokenizers google-generativeai openai diff-match-patch \
294
+ soundfile sounddevice
295
+
296
+ COPY . /app
297
+ EXPOSE 7860 7861
298
+ CMD ["python", "app.py"]
299
+ ```
300
+
301
+ **Why this layered install pattern works.** The pip resolver cannot satisfy
302
+ `pillow<12` (gradio) and `pillow==12.1.1` (aider) simultaneously. By
303
+ installing aider with `--no-deps` we let gradio's pillow 11 win. Aider only
304
+ imports a small fraction of its declared deps at runtime, and those that
305
+ matter are installed explicitly in Stage D. The litellm replacement is
306
+ necessary because the version aider pins (1.75.0) ships a broken module
307
+ surface β€” `litellm.APIConnectionError`, `litellm._logging`,
308
+ `litellm.encode`, and `litellm.token_counter` are all referenced by aider
309
+ but not exported by 1.75.0. 1.78.5 restores them.
310
+
311
+ ### 4.3 `requirements.txt` (resolved set)
312
+
313
+ The file declares: `requests`, `pytest`, `uv>=0.7.0`, `gitpython==3.1.46`,
314
+ `gradio>=5.49.0,<6`, `jinja2==3.1.6`, `ruff`, `tenacity`, `bandit[toml]`,
315
+ `pip-audit`, `radon`, `hypothesis[cli]>=6.100.0`, `semgrep>=1.45.0`,
316
+ `sentence-transformers>=2.7.0`, `sqlite-vec>=0.1.1`, `pygithub>=2.3.0`,
317
+ `PyJWT>=2.8.0`, `datasets>=2.19.0`, `numpy>=1.26.0`, `psycopg2-binary>=2.9.9`,
318
+ `rapidfuzz>=3.0.0`, `z3-solver>=4.12.0`, `qdrant-client>=1.9.0`,
319
+ `transformers>=4.40.0`, `torch>=2.2.0`, `starlette`, plus optional FastAPI
320
+ deps used by the Mythos API server.
321
+
322
+ `aider-chat` is intentionally absent β€” see Stage B above.
323
+
324
+ ---
325
+
326
+ ## 5. Process Map at Runtime
327
+ <a id="5-process-map-at-runtime"></a>
328
+
329
+ | Process / Thread | Started by | Lifetime | Listens on | Purpose |
330
+ |---|---|---|---|---|
331
+ | Main Python process | `python app.py` | container lifetime | β€” | imports modules, starts threads, hosts Gradio |
332
+ | Gradio Uvicorn worker | Gradio's `demo.launch()` | container lifetime | 7860 | serves the dashboard UI |
333
+ | Webhook Uvicorn worker | `webhook_server.start()` thread | container lifetime | 7861 | receives GitHub & CI webhooks |
334
+ | Audit threads | `submit_repo_audit()` per request | per audit | β€” | runs `enterprise_audit_loop` for one repo |
335
+ | Worker pool threads | `worker_pool.run()` inside an audit | per audit | β€” | parallel test repair |
336
+ | Worker subprocesses | optional, when `RHODAWK_PROCESS_ISOLATE=true` | per job | β€” | per-job blast radius isolation |
337
+ | Aider subprocess | `subprocess.Popen` per fix attempt | per attempt | β€” | LLM patch generation |
338
+ | MCP server processes | `npx`/`uvx` spawned by aider | per attempt | stdio | tools (fetch, github, filesystem, etc.) |
339
+ | Adversarial review threads | `concurrent.futures` in `adversarial_reviewer` | per gate | β€” | 3-model parallel review |
340
+ | Harvester daemon | `repo_harvester.start_daemon()` if enabled | container lifetime | β€” | autonomous target search |
341
+ | Commit watcher | `commit_watcher.start_daemon()` if enabled | container lifetime | β€” | silent-patch correlation |
342
+ | LoRA scheduler | `lora_scheduler.start_daemon()` if enabled | container lifetime | β€” | training data export trigger |
343
+ | Mythos FastAPI | optional | container lifetime | 7863 | external programmatic API |
344
+ | Leaderboard | optional | container lifetime | 7862 | public stats Gradio |
345
+
346
+ All daemons are gated by their respective `RHODAWK_*_ENABLED` env vars and
347
+ are off by default to keep the cold-start container minimal.
348
+
349
+ ---
350
+
351
+ ## 6. Master Orchestrator β€” `app.py`
352
+ <a id="6-master-orchestrator"></a>
353
+
354
+ `app.py` is 2,704 lines. It is the wiring file. Conceptually it has six
355
+ sections.
356
+
357
+ ### 6.1 Module imports & global config
358
+ Lines ~1–200. Imports every other module in the project. Reads
359
+ ~50 environment variables into module-level constants
360
+ (`DEFAULT_MODEL`, `MAX_RETRIES`, `WORKERS`, `OPENROUTER_API_KEY`,
361
+ `DO_INFERENCE_API_KEY`, `DO_INFERENCE_BASE_URL`, `DO_INFERENCE_MODEL`,
362
+ `FALLBACK_MODELS`, `RHODAWK_TENANT_ID`, audit/disclosure/training paths,
363
+ adversarial config, conviction thresholds, harvester config, etc.).
364
+
365
+ ### 6.2 MCP runtime config writer
366
+ Lines ~200–470. Reads `mcp_config.json` (the template), substitutes
367
+ `__INJECTED_BY_APP_AT_RUNTIME__` placeholders with real values from env,
368
+ and writes the materialized config to `/tmp/mcp_runtime.json`. Aider is
369
+ launched with `--mcp-config /tmp/mcp_runtime.json` so that all MCP tools
370
+ (fetch-docs, github-manager, filesystem-research, semgrep-sast, …) are
371
+ available during patch generation.
372
+
373
+ ### 6.3 `run_aider(prompt, repo_dir, model=None)` β€” provider chain
374
+ Lines ~600–700. The function described in Β§2.1. Chooses the provider list
375
+ based on which API keys are set, runs `aider` as a subprocess with the
376
+ right env and CLI flags, captures stdout/stderr, parses the diff, and
377
+ returns `(stdout, stderr, exit_code, model_used)`. On non-zero exit it
378
+ moves to the next provider.
379
+
380
+ ### 6.4 `process_failing_test(...)` β€” the 15-step healing loop
381
+ The single most important function in the project.
382
+
383
+ ```
384
+ 1. Memory retrieval β€” embedding_memory.retrieve_similar_fixes_v2(failure_text)
385
+ 2. Build prompt β€” verification_loop.build_initial_prompt(test_path, failure, similar)
386
+ 3. Aider call β€” run_aider(prompt, repo_dir) (provider chain)
387
+ 4. Re-run test β€” runtime.run_tests(test_path)
388
+ 5. If still FAIL β€” append failure delta, build_retry_prompt, GOTO 3 (up to MAX_RETRIES)
389
+ 6. SAST gate β€” sast_gate.run_sast(repo_dir, diff)
390
+ 7. If BLOCK β€” append SAST critique, GOTO 3
391
+ 8. Supply chain β€” supply_chain.run_supply_chain(diff, language)
392
+ 9. If BLOCK β€” append supply-chain critique, GOTO 3
393
+ 10. Z3 verify β€” formal_verifier.verify(diff) (if enabled)
394
+ 11. If UNSAFE β€” append Z3 critique, GOTO 3
395
+ 12. Adversarial β€” adversarial_reviewer.review_concurrent(diff, failure, repo)
396
+ 13. If REJECT β€” append adversarial critique, GOTO 3 (with extended budget)
397
+ 14. Persist β€” training_store.record_attempt(...)
398
+ 15. Return verdict β€” VerificationResult(success, attempts, verdicts...)
399
+ ```
400
+
401
+ The retry budget is normally `MAX_RETRIES` (default 5) but is multiplied by
402
+ `ADVERSARIAL_REJECTION_MULTIPLIER` when the retry was triggered by an
403
+ adversarial reject β€” to give the model more chances when the gate is the
404
+ strict adversarial trio rather than a hard test failure.
405
+
406
+ ### 6.5 `process_audit_test(...)` β€” wraps healing with PR + conviction
407
+ Calls `process_failing_test`, then on success:
408
+ - `github_app.open_pr_for_repo(repo, branch, title, body)` β†’ URL
409
+ - `conviction_engine.evaluate(verification_result, adversarial_result, memory_match)` β†’ `Conviction`
410
+ - If `Conviction.met` and `RHODAWK_AUTO_MERGE=true`: `github_app.merge_pr(url)`
411
+ - `audit_logger.append({event: PR_OPENED, ...})`
412
+ - `notifier.notify_pr_opened(repo, url)`
413
+
414
+ ### 6.6 `enterprise_audit_loop(repo)` β€” the per-audit driver
415
+ - Clones the repo into `/data/repo/<sanitized-owner>__<repo>`
416
+ - `RuntimeFactory.for_repo(repo_dir)` returns the right `LanguageRuntime`
417
+ - `runtime.setup_env()` (creates venv, installs deps)
418
+ - `runtime.discover_tests()` returns a list of test file paths
419
+ - Runs each test once to find the failing set
420
+ - Submits each failing test to the worker pool, which calls `process_audit_test`
421
+ - On completion: `red_team_fuzzer.start(repo_dir)` if all tests are now green
422
+ - Logs a final `AUDIT_COMPLETE` event
423
+
424
+ ### 6.7 Gradio UI
425
+ The `demo` block defines 10 tabs:
426
+ 1. **Chat Inbox** β€” submit `owner/repo` to start an audit
427
+ 2. **Live Agent Log** β€” real-time tail of the audit log with auto-refresh (3 s)
428
+ 3. **Audit Trail** β€” paginated view of `audit_trail.jsonl` with filters
429
+ 4. **Memory Browser** β€” search the embedding memory by failure text
430
+ 5. **Training Store** β€” stats and JSONL export trigger
431
+ 6. **Harvester** β€” refresh feed, view ranked candidates, dispatch manually
432
+ 7. **Findings** β€” vulnerability findings from Hermes / red team
433
+ 8. **Hermes Sessions** β€” list of research sessions with phases & VES scores
434
+ 9. **System Status** β€” env-var presence, daemon status, queue depth
435
+ 10. **Settings** β€” read-only display of effective configuration
436
+
437
+ The Live Agent Log uses a `gr.Timer` that fires every 3 seconds to repaint
438
+ the latest 200 lines of the in-memory log buffer.
439
+
440
+ ---
441
+
442
+ ## 7. Hermes Six-Phase Pipeline β€” `hermes_orchestrator.py`
443
+ <a id="7-hermes-pipeline"></a>
444
+
445
+ The Hermes orchestrator is a research-and-disclosure pipeline parallel to
446
+ the test-healing loop. It runs against a target codebase or live host and
447
+ produces structured `VulnerabilityFinding` records.
448
+
449
+ ### 7.1 Phases
450
+
451
+ | # | Enum value | Module(s) called | Output |
452
+ |---|---|---|---|
453
+ | 1 | `RECON` | `repo_harvester`, `mythos.mcp.reconnaissance_mcp`, `subdomain_enum_mcp`, `httpx_probe_mcp`, `wayback_mcp`, `shodan_mcp` | target inventory + tech fingerprint |
454
+ | 2 | `STATIC` | `taint_analyzer`, `semantic_extractor`, `mythos.static.{treesitter_cpg, joern_bridge, codeql_bridge, semgrep_bridge}` | candidate sinks + CWE hits |
455
+ | 3 | `DYNAMIC` | `fuzzing_engine`, `harness_factory`, `mythos.dynamic.{aflpp_runner, klee_runner, qemu_harness, frida_instr, gdb_automation}` | crashes, traces, coverage |
456
+ | 4 | `EXPLOIT` | `exploit_primitives`, `mythos.exploit.{pwntools_synth, rop_chain, heap_exploit, privesc_kb}` | working PoC artifacts |
457
+ | 5 | `CONSENSUS` | `adversarial_reviewer`, `_run_acts_consensus()` | model-quorum verdict + ACTS score |
458
+ | 6 | `DISCLOSURE` | `disclosure_vault`, `bounty_gateway`, `notifier` | encrypted writeup + optional submission |
459
+
460
+ ### 7.2 Tool dispatch
461
+ Each phase exposes a small set of `HermesTool` subclasses
462
+ (`ReconTool`, `TaintTool`, `SymbolicTool`, `FuzzTool`, `ExploitTool`,
463
+ `CVETool`, `CommitWatchTool`, `SSECTool`, `ChainAnalyzerTool`). The
464
+ `_dispatch_tool(tool_name, args, session)` function looks up the class by
465
+ name and runs it with the session context. Tool output is appended to
466
+ `session.findings` and persisted via `persist_hermes_session(session)`.
467
+
468
+ ### 7.3 Scoring
469
+ - `compute_ves(...)` β€” Vulnerability Evidence Score: weighted combination of
470
+ reproducibility, exploitability, impact, and code-confidence signals.
471
+ - `compute_acts(model_verdicts)` β€” Adversarial Consensus Trust Score:
472
+ measures inter-model agreement on the diff/exploit, used as an additional
473
+ gate before disclosure.
474
+
475
+ ### 7.4 LLM call surface
476
+ Every Hermes-triggered LLM call goes through `_hermes_llm_call`, which
477
+ implements the DO-primary / OpenRouter-fallback chain described in Β§2.2.
478
+ This means even adversarial review and disclosure-prose generation
479
+ benefit from the same provider resilience.
480
+
481
+ ### 7.5 Session persistence
482
+ `HermesSession` is a dataclass (`session_id`, `target`, `phase`,
483
+ `findings`, `tool_invocations`, `started_at`, `finished_at`,
484
+ `provider_history`, `ves`, `acts`). `persist_hermes_session(session)`
485
+ writes a JSON artifact under `/data/hermes_sessions/<session_id>.json`
486
+ and appends an audit-trail event. The Gradio "Hermes Sessions" tab reads
487
+ from this directory.
488
+
489
+ ### 7.6 TVG β€” Threat-Vector Graph
490
+ `build_tvg(repo_dir, findings)` produces a directed graph of
491
+ `(source β†’ sink, primitive, finding_id)` edges suitable for downstream
492
+ visualization or LLM-driven exploit-chain reasoning. Used by
493
+ `mythos/reasoning/attack_graph.py` to compute reachable attack chains.
494
+
495
+ ---
496
+
497
+ ## 8. Safety Gate Modules
498
+ <a id="8-safety-gates"></a>
499
+
500
+ ### 8.1 `sast_gate.py`
501
+ Runs **bandit** on Python diffs, **semgrep** with language-appropriate rule
502
+ sets on every diff, and a hand-rolled 16-pattern secret scanner over diff
503
+ additions only. Patterns include: AWS access keys (`AKIA[0-9A-Z]{16}`),
504
+ generic API keys (`api[_-]?key.*[=:].{20,}`), JWTs (`eyJ[\w-]+\.[\w-]+\.[\w-]+`),
505
+ private key headers (`-----BEGIN (RSA|EC|OPENSSH|PGP) PRIVATE KEY-----`),
506
+ `.env`-style assignments, OAuth tokens, SSH host keys, hardcoded passwords
507
+ in connection strings, etc. Returns `SastVerdict(level, findings, raw)`
508
+ where `level ∈ {PASS, BLOCK}`. A single CRITICAL or HIGH finding flips to
509
+ BLOCK; MEDIUM/LOW are surfaced but advisory.
510
+
511
+ ### 8.2 `supply_chain.py`
512
+ Two passes per diff:
513
+ 1. **Known CVEs.** Extract added packages from `requirements.txt`,
514
+ `package.json`, `pom.xml`, `go.mod`, `Cargo.toml`, `Gemfile`. Run the
515
+ matching CVE scanner: `pip-audit`, `npm audit --json`, OWASP
516
+ `dependency-check`, `govulncheck`, `cargo audit`, `bundler-audit`.
517
+ 2. **Typosquatting.** For each new package name, compute Levenshtein
518
+ distance against a 40+ entry list of canonical popular package names.
519
+ Anything within distance 1 of a popular name (and not equal to it)
520
+ is flagged as a likely typosquat.
521
+
522
+ ### 8.3 `formal_verifier.py`
523
+ Three Z3-backed checks over diff additions:
524
+ - **Div-by-zero.** Pattern-extract `expr / divisor` and `expr % divisor`,
525
+ parse `divisor` as a Z3 expr, query `solver.check(divisor == 0)`.
526
+ - **Negative literal index.** Find `arr[k]` where `k` is an integer literal,
527
+ query satisfiability of `k < 0`.
528
+ - **Always-false assert.** For each `assert expr`, walk the surrounding
529
+ scope for `name = literal` constants, substitute, query
530
+ `solver.check(Not(expr))`.
531
+ Returns `FormalVerdict(SAFE | UNSAFE | SKIP, witnesses)`. Disabled by
532
+ default (`RHODAWK_Z3_ENABLED=false`) because Z3 is a heavy import.
533
+
534
+ ### 8.4 `adversarial_reviewer.py`
535
+ Sends the diff + failure + repo identity to three reviewer models in
536
+ parallel via `concurrent.futures.ThreadPoolExecutor(max_workers=3)`. Each
537
+ model receives an instruction template that demands a strict JSON envelope:
538
+ `{verdict, confidence, critical_issues[], warnings[], retry_guidance}`.
539
+ The reviewer parses each envelope, computes:
540
+ - `consensus_votes = Counter(verdicts)`
541
+ - `consensus_fraction = max(votes.values()) / 3`
542
+ - final verdict = majority (β‰₯`RHODAWK_CONSENSUS_THRESHOLD` of 0.67)
543
+ - `critical_issues = union(per-model critical lists)`
544
+
545
+ Sequential mode (`RHODAWK_ADVERSARY_SEQUENTIAL=true`) runs the same models
546
+ serially. Used as a fallback when concurrent calls hit OpenRouter's
547
+ per-key concurrency limits.
548
+
549
+ ### 8.5 `conviction_engine.py`
550
+ Seven boolean criteria, all of which must be true to allow autonomous
551
+ merge:
552
+ 1. `adversarial.verdict == APPROVE` (not just CONDITIONAL)
553
+ 2. `adversarial.confidence β‰₯ RHODAWK_CONVICTION_CONFIDENCE` (default 0.92)
554
+ 3. `adversarial.consensus_fraction β‰₯ RHODAWK_CONVICTION_CONSENSUS` (0.85)
555
+ 4. `verification.attempts == 1` (fixed on first try)
556
+ 5. `sast.findings == []` (zero SAST findings)
557
+ 6. `supply_chain.added_packages == []` (no new deps)
558
+ 7. `memory.best_match.similarity β‰₯ RHODAWK_CONVICTION_MEMORY_SIM` AND
559
+ `memory.best_match.merged_by_human == True`
560
+
561
+ Returns `Conviction(met: bool, missing_criteria: list, score: float)`.
562
+ The `score` is a weighted combination useful for analytics even when
563
+ `met` is False.
564
+
565
+ ### 8.6 `taint_analyzer.py`
566
+ Lightweight per-language source/sink dataflow analyzer used during the
567
+ Hermes STATIC phase (not in the patch healing loop). Tracks tainted
568
+ variables from common sources (HTTP request data, file reads, env vars,
569
+ CLI args) to dangerous sinks (eval, subprocess, raw SQL, `pickle.loads`,
570
+ `yaml.load`, `etree.fromstring`).
571
+
572
+ ### 8.7 `symbolic_engine.py`
573
+ Wraps Z3 for richer symbolic execution beyond the single-statement checks
574
+ in `formal_verifier.py`. Used by Hermes `SymbolicTool`.
575
+
576
+ ### 8.8 `vuln_classifier.py`
577
+ Maps raw findings (from SAST tools, taint, fuzzers) to CWE IDs and
578
+ estimated CVSS using a rule table. Output feeds `disclosure_vault` and
579
+ `bounty_gateway`.
580
+
581
+ ---
582
+
583
+ ## 9. Memory, Learning, and Data Flywheel
584
+ <a id="9-memory-learning"></a>
585
+
586
+ ### 9.1 `embedding_memory.py`
587
+ Two interchangeable backends:
588
+
589
+ **SQLite + MiniLM (default).** `sentence-transformers/all-MiniLM-L6-v2` for
590
+ encoding. Embeddings stored in a SQLite table `(id, failure_norm, embedding
591
+ BLOB, fix_diff, outcome, repo, ts)` and queried by cosine similarity using
592
+ `sqlite-vec`. Lightweight, runs entirely in-process, no external service.
593
+
594
+ **Qdrant + CodeBERT (optional).** `microsoft/codebert-base` for code-aware
595
+ encoding. Vectors pushed to a Qdrant collection (HNSW index) β€” local or
596
+ remote. Switched on via `RHODAWK_EMBEDDING_BACKEND=qdrant`.
597
+
598
+ Public API used by the rest of the system:
599
+ - `retrieve_similar_fixes_v2(failure_text, top_k=5)` β†’ list of past fixes
600
+ - `record_fix(failure_text, diff, outcome, repo)` β†’ persists for future retrieval
601
+ - `rebuild_embedding_index()` β†’ re-encode entire training_store on demand
602
+
603
+ Failure normalization strips file paths, line numbers, addresses, and
604
+ hex IDs so that semantically identical failures across different repos
605
+ collide in embedding space.
606
+
607
+ ### 9.2 `memory_engine.py`
608
+ Legacy v1 store. Exact-key lookup by normalized failure signature. Kept
609
+ for cold-start scenarios where the embedding index has zero entries.
610
+
611
+ ### 9.3 `training_store.py`
612
+ SQLite database (`/data/training_store.sqlite`) with two tables:
613
+ - `fix_attempts(id, ts, repo, test_path, failure, diff, model, attempt, success)`
614
+ - `fix_patterns(signature, success_count, attempt_count, last_seen)`
615
+
616
+ Entry points:
617
+ - `record_attempt(...)` β€” every fix attempt, success or failure
618
+ - `export_training_data(jsonl_path)` β€” writes
619
+ `{"messages": [{"role": "system", ...}, {"role": "user", ...},
620
+ {"role": "assistant", "content": diff}]}` β€” directly compatible with
621
+ HuggingFace TRL/PEFT SFT trainers.
622
+ - `get_statistics()` β€” counts for the dashboard
623
+
624
+ ### 9.4 `lora_scheduler.py`
625
+ Background daemon. Polls the training store. When **either**
626
+ `RHODAWK_LORA_MIN_SAMPLES` (default 50) new successful fix pairs accumulate
627
+ **or** `RHODAWK_LORA_MAX_AGE_HOURS` (default 168 = 1 week) elapse since the
628
+ last export, it calls `training_store.export_training_data()` to a new
629
+ JSONL file under `/data/lora_exports/lora_<timestamp>.jsonl`.
630
+
631
+ ### 9.5 `knowledge_rag.py`
632
+ Retrieval-augmented generation helper for Hermes. Indexes long-form security
633
+ documentation (the `architect/skills/*.md` library plus optionally fetched
634
+ CVE writeups) and exposes a `retrieve(query, top_k)` helper that the
635
+ research planner can call to ground exploit reasoning in vetted prior art.
636
+
637
+ ---
638
+
639
+ ## 10. Autonomous Operation β€” Harvester, Worker Pool, Job Queue
640
+ <a id="10-autonomous-operation"></a>
641
+
642
+ ### 10.1 `repo_harvester.py`
643
+ - Background daemon, gated by `RHODAWK_HARVESTER_ENABLED=true`.
644
+ - Cycle period: `RHODAWK_HARVESTER_POLL_SECONDS` (default 21600 = 6 h).
645
+ - Per cycle: GitHub Search API for repos in 7 supported languages
646
+ (`language:python`, `language:javascript`, …) with at least
647
+ `RHODAWK_HARVESTER_MIN_STARS` (default 100) and recent activity.
648
+ - Cross-references each candidate's `check-runs` API to find currently
649
+ failing CI.
650
+ - Scoring: `0.35*log(stars+1)/log(MAX_STARS+1) + 0.40*recency_score +
651
+ 0.25*failing_check_count_score`.
652
+ - Persists ranked feed to `/data/harvester_feed.json`.
653
+ - Dispatches the top `RHODAWK_HARVESTER_MAX_REPOS` (default 20) targets
654
+ into the audit loop via `submit_repo_audit()`.
655
+
656
+ ### 10.2 `worker_pool.py`
657
+ - `ThreadPoolExecutor(max_workers=RHODAWK_WORKERS)` (default 8).
658
+ - When `RHODAWK_PROCESS_ISOLATE=true`: each job runs in
659
+ `multiprocessing.get_context("fork").Process` with a result `Queue`
660
+ and a hard timeout of `RHODAWK_ISOLATE_TIMEOUT` seconds (default 600).
661
+ - On timeout: process is `terminate()`d then `kill()`ed if still alive,
662
+ job is marked `FAILED`, audit-trail event recorded.
663
+ - Falls back gracefully to in-process execution if `fork` is not available
664
+ (e.g., Windows host β€” irrelevant to HF but matters for self-hosters).
665
+
666
+ ### 10.3 `job_queue.py`
667
+ - SQLite-backed (`/data/job_queue.sqlite`).
668
+ - Schema: `(tenant_id, repo, test_path)` UNIQUE β†’ `(status, pr_url,
669
+ model_version, started_at, updated_at, attempts, last_error)`.
670
+ - States: `PENDING`, `RUNNING`, `DONE`, `FAILED`, `SAST_BLOCKED`,
671
+ `ADVERSARIAL_REJECTED`, `CONVICTION_NOT_MET`.
672
+ - Idempotency: a `DONE` row is skipped on resubmission. A `RUNNING` row
673
+ older than 1 hour (stale, from a crashed previous container) is reset to
674
+ `PENDING` after a branch-cleanup pass.
675
+ - `prune_done_jobs(older_than_hours=72)` keeps the table from growing
676
+ unbounded.
677
+
678
+ ---
679
+
680
+ ## 11. GitHub Surface β€” `github_app.py`, `webhook_server.py`, `commit_watcher.py`
681
+ <a id="11-github-surface"></a>
682
+
683
+ ### 11.1 `github_app.py`
684
+ - **PAT mode (default).** Uses `GITHUB_TOKEN` directly via PyGithub.
685
+ - **GitHub App mode.** Uses `RHODAWK_APP_ID` + `RHODAWK_APP_PRIVATE_KEY`
686
+ to issue a 10-minute JWT, exchange it for a 1-hour installation token,
687
+ and refresh on each call. Enterprise-grade auth.
688
+ - `open_pr_for_repo(repo, branch, title, body)` β€” chooses standard or
689
+ fork mode based on `RHODAWK_FORK_MODE`.
690
+ - **Standard mode.** Push branch to upstream, open PR upstream→upstream.
691
+ - **Fork mode.** `fork_repo(repo)` (waits for GitHub's async fork to
692
+ complete by polling), push branch to fork, open cross-repo PR
693
+ fork→upstream. This is what enables fixing repos you don't own.
694
+ - `merge_pr(url)` β€” used by the conviction engine when criteria are met.
695
+
696
+ ### 11.2 `webhook_server.py`
697
+ HTTP server on port 7861. Endpoints:
698
+ - `POST /webhook/github` β€” validates `X-Hub-Signature-256` HMAC against
699
+ `RHODAWK_WEBHOOK_SECRET`. Handles `push` and `check_run` events.
700
+ - `POST /webhook/ci` β€” generic CI failure trigger (token in header).
701
+ - `POST /webhook/trigger` β€” manual trigger (admin token).
702
+ - `GET /webhook/health` β€” liveness.
703
+ - `GET /webhook/queue` β€” current queue depth + last 20 events.
704
+ Per-IP token bucket rate limiter (default 30 req/min per IP).
705
+
706
+ ### 11.3 `commit_watcher.py`
707
+ Daemon for **silent-patch detection**. Polls a configured set of
708
+ high-value upstream repos for new commits. Heuristics flag commits whose
709
+ message says "fix" / "security" / "CVE" but which have no associated
710
+ public advisory β€” a known pattern for CAD (Coordinated Asynchronous
711
+ Disclosure) or under-the-radar security fixes. Findings are routed to
712
+ `disclosure_vault` for downstream attribution.
713
+
714
+ ### 11.4 `chain_analyzer.py`
715
+ Walks PR commit chains and groups related commits into "fix series" so
716
+ that the training store records the *full* fix, not just the first commit
717
+ in a multi-commit fix. Important for accurate (failure β†’ fix) pair
718
+ generation.
719
+
720
+ ---
721
+
722
+ ## 12. Red Team & Offense Modules
723
+ <a id="12-red-team--offense"></a>
724
+
725
+ ### 12.1 `red_team_fuzzer.py`
726
+ 1,561 lines. The Blue Team-to-Red Team transition. When
727
+ `process_audit_test` finishes and all tests are green, this module:
728
+ - Generates property-based tests using **Hypothesis** strategies inferred
729
+ from function signatures.
730
+ - Runs them with `pytest --hypothesis-show-statistics`.
731
+ - Captures shrunken counterexamples.
732
+ - Feeds the counterexamples into a CEGIS loop: each refuted property
733
+ becomes the spec for the next attack synthesis round.
734
+ - Zero-day candidates are written as JSON artifacts under `/data/findings/`
735
+ and handed back to the Blue Team healing loop so the bug becomes a fix.
736
+
737
+ ### 12.2 `fuzzing_engine.py`
738
+ Lower-level fuzzing primitives consumed by both `red_team_fuzzer.py` and
739
+ the Hermes DYNAMIC phase. Bridges to `mythos/dynamic/aflpp_runner.py`,
740
+ `klee_runner.py`, and `qemu_harness.py` when those are available.
741
+
742
+ ### 12.3 `harness_factory.py`
743
+ Generates fuzz harnesses from function signatures: type inference,
744
+ seed corpus extraction from existing tests, AFL++/libFuzzer wrapper
745
+ emission. Used by `red_team_fuzzer.py` and Hermes.
746
+
747
+ ### 12.4 `exploit_primitives.py`
748
+ Catalogs reusable exploitation primitives: arbitrary read/write,
749
+ function-pointer overwrite, format-string leak, integer-overflow ladder,
750
+ heap-grooming templates. Each primitive has a `match(finding) β†’ bool`
751
+ predicate and an `apply(finding) β†’ ExploitArtifact` synthesizer.
752
+
753
+ ### 12.5 `cve_intel.py`
754
+ NVD/OSV/Snyk lookup for prior-art correlation. Given a finding, returns
755
+ the closest known CVE and similarity score so the disclosure isn't a
756
+ duplicate.
757
+
758
+ ---
759
+
760
+ ## 13. Notification, Audit, Disclosure, Bounty
761
+ <a id="13-notification-audit"></a>
762
+
763
+ ### 13.1 `notifier.py`
764
+ - Telegram via Bot API (`TELEGRAM_BOT_TOKEN` + `TELEGRAM_CHAT_ID`).
765
+ - Slack via incoming webhook (`SLACK_WEBHOOK_URL`).
766
+ - All sends are non-blocking (background threads with bounded queue).
767
+ - Templates for: AUDIT_START, TEST_FAIL, PR_CREATED, PATCH_FAIL,
768
+ SAST_BLOCK, AUDIT_COMPLETE, ZERO_DAY_FOUND, DISCLOSURE_SUBMITTED.
769
+
770
+ ### 13.2 `audit_logger.py`
771
+ Append-only JSONL at `/data/audit_trail.jsonl`. Each entry:
772
+ ```json
773
+ {
774
+ "ts": "2026-04-22T12:34:56Z",
775
+ "event": "ADVERSARIAL_REVIEW",
776
+ "tenant": "acme",
777
+ "repo": "owner/name",
778
+ "payload": {...},
779
+ "prev_hash": "sha256:...",
780
+ "hash": "sha256:..."
781
+ }
782
+ ```
783
+ `prev_hash` is the hash of the previous entry; `hash` is the SHA-256 of
784
+ the current entry (excluding the `hash` field itself). The genesis entry
785
+ has `prev_hash = "0"*64`.
786
+
787
+ `verify_chain_integrity()` walks the file from genesis, recomputes each
788
+ hash, and asserts equality. Any tamper anywhere breaks the chain at the
789
+ tampered point and every subsequent entry. SOC 2 / ISO 27001 evidence
790
+ artifact.
791
+
792
+ ### 13.3 `disclosure_vault.py`
793
+ Encrypted at-rest storage for vulnerability writeups. Uses Fernet with a
794
+ key derived from `DISCLOSURE_VAULT_PASSPHRASE`. Each entry:
795
+ finding metadata + full PoC + suggested fix + CVE correlation. Read API
796
+ requires the same passphrase. Used by Hermes DISCLOSURE phase before
797
+ any public submission.
798
+
799
+ ### 13.4 `bounty_gateway.py`
800
+ Programmatic submission to HackerOne, Bugcrowd, Intigriti, YesWeHack via
801
+ their REST APIs. Submission requires:
802
+ - Finding has passed adversarial CONSENSUS phase
803
+ - VES β‰₯ configured floor
804
+ - Manual approval flag (default: required) OR
805
+ `RHODAWK_BOUNTY_AUTO_SUBMIT=true` (off by default β€” kept off in
806
+ production)
807
+
808
+ ### 13.5 `oss_guardian.py`
809
+ Stewardship layer for fixes pushed to OSS repos. Tracks PR status,
810
+ maintainer responses, and time-to-merge. Feeds analytics for the public
811
+ leaderboard and for the conviction engine's "merged by human" signal.
812
+
813
+ ### 13.6 `oss_target_scorer.py`
814
+ Scoring helper used by both the harvester and the public leaderboard to
815
+ rank repos by community impact (stars, dependents, ecosystem centrality).
816
+
817
+ ### 13.7 `public_leaderboard.py`
818
+ Standalone Gradio interface (port 7862) showing PRs submitted, PRs
819
+ merged, repos touched, patterns learned, zero-days reported. All numbers
820
+ read directly from `audit_trail.jsonl` and `training_store.sqlite` β€”
821
+ no synthetic metrics.
822
+
823
+ ### 13.8 `swebench_harness.py`
824
+ Runs the Rhodawk healing loop against SWE-bench Verified instances.
825
+ Reports pass@1. Same pipeline as production β€” no special-cased logic.
826
+ Provides reproducible benchmarks for investor/customer claims.
827
+
828
+ ---
829
+
830
+ ## 14. Architect Subsystem β€” Tier Routing, Sandbox, Skills, Night Mode
831
+ <a id="14-architect"></a>
832
+
833
+ The `architect/` package is the higher-order control layer that sits above
834
+ the per-test healing loop and decides which model, which skill, and which
835
+ sandbox each task should use.
836
+
837
+ ### 14.1 `architect/model_router.py`
838
+ Five-tier model router. Each task is classified by complexity and routed
839
+ to the cheapest tier that can satisfy it:
840
+ - **Tier 1 β€” Ultra.** DeepSeek 3.2, MiniMax 2.5. Used for global strategy
841
+ and multi-repo planning.
842
+ - **Tier 2 β€” Strong.** Qwen 2.5 Coder 32B. Default patch generator.
843
+ - **Tier 3 β€” Balanced.** Llama 3.3 70B (DigitalOcean), DeepSeek V3,
844
+ Gemma 2 27B. Adversarial reviewers.
845
+ - **Tier 4 β€” Lite.** Mistral 7B, Gemma 2 9B. Cheap second-opinion calls.
846
+ - **Tier 5 β€” Local.** Embedded MiniLM/CodeBERT for embeddings β€” no LLM call.
847
+
848
+ The router consumes `(task_type, code_size_tokens, security_sensitivity,
849
+ required_capabilities)` and returns a routing decision with `provider`,
850
+ `model`, and `expected_cost_usd`.
851
+
852
+ ### 14.2 `architect/sandbox.py`
853
+ `SandboxManager` β€” abstraction over (a) plain subprocess, (b) `bwrap`
854
+ (bubblewrap) namespace isolation, (c) Firecracker microVM (planned).
855
+ Currently used in subprocess + bwrap modes for any tool execution that
856
+ must not touch the host filesystem outside `/tmp/sandbox/<id>/`.
857
+
858
+ ### 14.3 `architect/skill_registry.py`
859
+ Registers the 28 skill markdown files under `architect/skills/` as
860
+ discoverable knowledge units. Each skill file declares: domain
861
+ (api-security, container-escape, smart-contract-audit, …), required
862
+ tools, suggested model tier, and reference references. The registry
863
+ exposes `find_skills(query) β†’ list[Skill]` for the planner.
864
+
865
+ ### 14.4 `architect/skills/`
866
+ 28 long-form security playbooks covering: api-security, ai-ml-security,
867
+ automotive-security, aviation-aerospace, binary-analysis, browser-engine
868
+ security, ci-cd-pipeline-attack, cloud-security, container-escape,
869
+ cryptographic-implementation, cryptography-attacks, firmware-analysis,
870
+ hardware-protocols, ics-scada, linux-kernel-exploitation, llm-system-
871
+ prompt-injection, memory-safety, mobile-android, mobile-ios, network-
872
+ protocol, reverse-engineering, rf-radio-security, satellite-comms,
873
+ smart-contract-audit, supply-chain, vibe-coded-app-hunter, web-security
874
+ advanced, zero-day-research, plus `bb-methodology-claude.md` and
875
+ `bug-bounty-reference-index.md`.
876
+
877
+ ### 14.5 `architect/nightmode.py`
878
+ The autonomous "night mode" loop. When enabled, between human-driven
879
+ audits the system pulls in scope from connected bug-bounty platforms
880
+ (via `mythos.mcp.scope_parser_mcp`), enumerates targets, and runs
881
+ Hermes pipelines against authorized scope only β€” submitting findings
882
+ through `bounty_gateway`. Hard-gated by `ARCHITECT_NIGHTMODE_ENABLED=true`
883
+ and a per-platform scope-acceptance check.
884
+
885
+ ### 14.6 `architect/godmode_consensus.py`
886
+ A heavier-weight consensus protocol used by night mode that runs the
887
+ adversarial trio AND a Tier 1 model, requiring unanimous agreement
888
+ before any external submission. Higher false-negative rate, near-zero
889
+ false-positive rate β€” appropriate when the action is irreversible.
890
+
891
+ ### 14.7 `architect/master_redteam_prompt.py`
892
+ Centralizes the master prompt used by the red-team modules. Externalizing
893
+ the prompt makes it tunable without code changes and auditable for
894
+ prompt-injection resistance.
895
+
896
+ ### 14.8 `architect/parseltongue.py`
897
+ Internal DSL for declaring multi-step research plans. Compiles to a
898
+ sequence of `HermesTool` invocations. Used by the Mythos planner agent.
899
+
900
+ ### 14.9 `architect/embodied_bridge.py`
901
+ Bridge to the EmbodiedOS / OpenClaw / Hermes Agent runtime when present
902
+ externally. Stubs gracefully when EmbodiedOS is not reachable.
903
+
904
+ ### 14.10 `architect/rl_feedback_loop.py`
905
+ RL feedback collector. After each fix, an outcome reward is computed
906
+ (merge=+1, rejected=-0.5, never-reviewed-after-30-days=-0.1) and the
907
+ plan-level statistics are updated for the planner's policy table.
908
+
909
+ ---
910
+
911
+ ## 15. Mythos Subsystem β€” Multi-Agent + RL + 17 MCP Servers
912
+ <a id="15-mythos"></a>
913
+
914
+ `mythos/` is the deepest research layer β€” designed to close the gap between
915
+ Rhodawk's deterministic 6-phase pipeline and a Mythos-class autonomous
916
+ research agent.
917
+
918
+ ### 15.1 `mythos/MYTHOS_PLAN.md`
919
+ Living blueprint document mapping Rhodawk's gaps to concrete Mythos
920
+ modules. Read this first to understand the design intent.
921
+
922
+ ### 15.2 `mythos/agents/`
923
+ - `base.py` β€” abstract `Agent` class with `observe`, `plan`, `act`.
924
+ - `planner.py` β€” generates research plans from a target description.
925
+ Uses the Tier 1 model + RL policy table.
926
+ - `explorer.py` β€” enumeration agent for recon and surface mapping.
927
+ - `executor.py` β€” runs concrete tools (calls into `mythos/static`,
928
+ `mythos/dynamic`, `mythos/exploit`).
929
+ - `orchestrator.py` β€” top-level coordinator that wires planner β†’
930
+ explorer β†’ executor β†’ consensus and persists the trace.
931
+
932
+ ### 15.3 `mythos/static/`
933
+ - `treesitter_cpg.py` β€” Tree-sitter–based Code Property Graph builder.
934
+ - `joern_bridge.py` β€” Joern CPG queries for taint chains.
935
+ - `codeql_bridge.py` β€” CodeQL pack runner with predefined query sets.
936
+ - `semgrep_bridge.py` β€” Semgrep with curated security rule packs.
937
+
938
+ ### 15.4 `mythos/dynamic/`
939
+ - `aflpp_runner.py` β€” AFL++ harness execution.
940
+ - `klee_runner.py` β€” KLEE symbolic execution wrapper.
941
+ - `qemu_harness.py` β€” full-system QEMU with snapshot/restore.
942
+ - `frida_instr.py` β€” Frida runtime instrumentation sessions.
943
+ - `gdb_automation.py` β€” scripted GDB triage of crashes.
944
+
945
+ ### 15.5 `mythos/exploit/`
946
+ - `pwntools_synth.py` β€” pwntools-based PoC synthesis.
947
+ - `rop_chain.py` β€” ROP gadget search + chain assembly via ROPGadget.
948
+ - `heap_exploit.py` β€” heap exploitation primitives (tcache, fastbin,
949
+ unsafe-unlink).
950
+ - `privesc_kb.py` β€” privilege escalation knowledge base + matcher.
951
+
952
+ ### 15.6 `mythos/learning/`
953
+ - `rl_planner.py` β€” RL planner using the reward signal from
954
+ `architect/rl_feedback_loop.py`.
955
+ - `curriculum.py` β€” curriculum learning over progressively harder
956
+ vulnerability classes.
957
+ - `episodic_memory.py` β€” episodic store of full
958
+ (target, plan, action, observation, reward) trajectories.
959
+ - `mlflow_tracker.py` β€” MLflow run tracking for training experiments.
960
+ - `lora_adapters.py` β€” LoRA adapter loader; lets the system swap in
961
+ domain-specific adapters per task.
962
+
963
+ ### 15.7 `mythos/reasoning/`
964
+ - `probabilistic.py` β€” probabilistic attack-vector reasoning.
965
+ - `attack_graph.py` β€” attack-graph construction over `(asset, action,
966
+ result)` triples; computes minimum-cost reachable goal paths.
967
+
968
+ ### 15.8 `mythos/api/`
969
+ - `fastapi_server.py` β€” optional FastAPI server (port 7863) exposing
970
+ `/audit`, `/research`, `/findings`, `/leaderboard` JSON APIs.
971
+ - `auth.py` β€” bearer-token auth middleware.
972
+ - `schemas.py` β€” Pydantic request/response models.
973
+ - `webhooks.py` β€” webhook delivery on async events.
974
+
975
+ ### 15.9 `mythos/skills/`
976
+ - `registry.py` β€” runtime skill discovery and ranking. Bridges to
977
+ `architect/skill_registry.py` to expose skills via MCP.
978
+
979
+ ### 15.10 `mythos/integration.py`
980
+ Top-level wiring. Initializes the Mythos subsystem from `app.py`'s
981
+ startup. If Mythos modules fail to import (optional dependencies
982
+ missing), it logs a warning and lets the system continue without the
983
+ Mythos features.
984
+
985
+ ### 15.11 `mythos/diagnostics.py`
986
+ Self-test CLI. `python -m mythos.diagnostics` prints which agents,
987
+ MCP servers, and external tools are reachable, and exits non-zero if
988
+ any required component is broken. Used by the `mythos.__main__` entry
989
+ point and by container health checks.
990
+
991
+ ### 15.12 `mythos/__main__.py`
992
+ Allows `python -m mythos` to launch the Mythos subsystem standalone.
993
+
994
+ ---
995
+
996
+ ## 16. MCP Suite β€” Every Server, Every Domain
997
+ <a id="16-mcp-suite"></a>
998
+
999
+ The `mcp_config.json` template declares **34 MCP server entries** (with
1000
+ some duplicates β€” see note below). Aider is launched with the rendered
1001
+ `/tmp/mcp_runtime.json` so all of these tools become callable during
1002
+ patch generation. Hermes also calls them directly.
1003
+
1004
+ ### 16.1 Generic / shared servers
1005
+ | Server | Command | Purpose |
1006
+ |---|---|---|
1007
+ | `fetch-docs` | `uvx mcp-server-fetch` | HTTP fetch with SSRF allow-list (60+ domains) |
1008
+ | `github-manager` | `npx @modelcontextprotocol/server-github` | full GitHub API |
1009
+ | `filesystem-research` | `npx @modelcontextprotocol/server-filesystem` | RO access to `/data/repo`, `/tmp/research`, `/tmp/findings` |
1010
+ | `memory-store` | `npx @modelcontextprotocol/server-memory` | persistent KG of exploit chains |
1011
+ | `sequential-thinking` | `npx @modelcontextprotocol/server-sequential-thinking` | structured CoT |
1012
+ | `web-search` | `npx @modelcontextprotocol/server-brave-search` | Brave search API |
1013
+ | `git-forensics` | `npx @modelcontextprotocol/server-git` | deep git-history analysis |
1014
+ | `postgres-intelligence` | `npx @modelcontextprotocol/server-postgres` | findings DB queries |
1015
+ | `sqlite-findings` | `npx @modelcontextprotocol/server-sqlite` | local findings DB |
1016
+
1017
+ ### 16.2 Security tooling shells (`mcp-server-shell` allow-list)
1018
+ | Server | Allowed binaries | Purpose |
1019
+ |---|---|---|
1020
+ | `nuclei-scanner` | `nuclei,nuclei-templates` | DAST + CVE templates |
1021
+ | `semgrep-sast` | `semgrep` | taint + CWE + secrets |
1022
+ | `trufflehog-secrets` | `trufflehog` | git-history secret scan |
1023
+ | `bandit-sast` | `bandit` | Python AST SAST |
1024
+ | `pip-audit-sca` | `pip-audit,pip` | OSV-backed Python SCA |
1025
+ | `osv-scanner` | `osv-scanner` | multi-ecosystem SCA |
1026
+ | `z3-formal-verifier` | `python3` | Z3 verification scripts |
1027
+ | `hypothesis-fuzzer` | `python3,pytest,hypothesis` | property-based testing |
1028
+ | `atheris-fuzzer` | `python3,atheris` | coverage-guided fuzzing |
1029
+ | `angr-symbolic` | `python3` | binary symbolic execution |
1030
+ | `radon-complexity` | `radon` | complexity / attack surface |
1031
+ | `ruff-linter` | `ruff` | anti-pattern detection |
1032
+ | `aider-patcher` | `aider` | patch synthesis (recursive β€” careful) |
1033
+
1034
+ ### 16.3 Intelligence + bounty
1035
+ | Server | Purpose |
1036
+ |---|---|
1037
+ | `cve-intelligence` | NVD / CVE.org / OSV fetch |
1038
+ | `bounty-platform` | HackerOne/Bugcrowd/Intigriti/YesWeHack APIs |
1039
+ | `supply-chain-monitor` | PyPI/npm/crates typosquatting + dep-confusion |
1040
+
1041
+ ### 16.4 Mythos-native MCP servers (Python modules)
1042
+ | Server | Module | Purpose |
1043
+ |---|---|---|
1044
+ | `reconnaissance-mcp` | `mythos.mcp.reconnaissance_mcp` | language/framework/dep fingerprinting + attack surface |
1045
+ | `static-analysis-mcp` | `mythos.mcp.static_analysis_mcp` | Tree-sitter CPG + Joern + CodeQL + Semgrep |
1046
+ | `dynamic-analysis-mcp` | `mythos.mcp.dynamic_analysis_mcp` | AFL++ + KLEE + QEMU + Frida + GDB |
1047
+ | `exploit-generation-mcp` | `mythos.mcp.exploit_generation_mcp` | Pwntools + ROP + heap + privesc |
1048
+ | `vulnerability-database-mcp` | `mythos.mcp.vulnerability_database_mcp` | NVD/OSV/Exploit-DB lookup |
1049
+ | `web-security-mcp` | `mythos.mcp.web_security_mcp` | OWASP ZAP + nuclei + sqlmap orchestration |
1050
+ | `browser-agent-mcp` | `mythos.mcp.browser_agent_mcp` | Playwright live browser |
1051
+ | `scope-parser-mcp` | `mythos.mcp.scope_parser_mcp` | bug-bounty scope ingestion |
1052
+ | `subdomain-enum-mcp` | `mythos.mcp.subdomain_enum_mcp` | subfinder + amass + dnsx + crt.sh |
1053
+ | `httpx-probe-mcp` | `mythos.mcp.httpx_probe_mcp` | concurrent HTTP probing + tech fingerprint |
1054
+ | `shodan-mcp` | `mythos.mcp.shodan_mcp` | Shodan REST passive recon |
1055
+ | `wayback-mcp` | `mythos.mcp.wayback_mcp` | Wayback / CommonCrawl historical URL recall |
1056
+ | `frida-runtime-mcp` | `mythos.mcp.frida_runtime_mcp` | live Frida instrumentation |
1057
+ | `ghidra-bridge-mcp` | `mythos.mcp.ghidra_bridge_mcp` | headless Ghidra / radare2 bridge |
1058
+ | `can-bus-mcp` | `mythos.mcp.can_bus_mcp` | automotive CAN-bus + UDS (ISO 14229) |
1059
+ | `sdr-analysis-mcp` | `mythos.mcp.sdr_analysis_mcp` | GNU Radio / rtl_sdr RF capture |
1060
+
1061
+ > **Duplicate-key note.** `mcp_config.json` currently declares
1062
+ > `scope-parser-mcp`, `subdomain-enum-mcp`, `wayback-mcp`, `httpx-probe-mcp`,
1063
+ > and `shodan-mcp` twice. JSON parse semantics use the last value, so the
1064
+ > ARCHITECT-flavored variants (with `env` injections for HackerOne/Shodan
1065
+ > tokens) are what wins at runtime. The earlier entries are dead weight
1066
+ > and could be removed in a future cleanup commit.
1067
+
1068
+ ### 16.5 Runtime materialization
1069
+ On startup, `app.py` reads `mcp_config.json`, recursively replaces every
1070
+ `__INJECTED_BY_APP_AT_RUNTIME__` placeholder with the corresponding env
1071
+ var value (`OPENROUTER_API_KEY`, `BRAVE_API_KEY`, `DATABASE_URL`,
1072
+ `HACKERONE_API_TOKEN`, `BUGCROWD_API_TOKEN`, `INTIGRITI_API_TOKEN`,
1073
+ `SHODAN_API_KEY`, `NVD_API_KEY`, `NUCLEI_API_KEY`, `SEMGREP_APP_TOKEN`,
1074
+ `GITHUB_PERSONAL_ACCESS_TOKEN`), drops any server whose required secret
1075
+ is missing, and writes the result to `/tmp/mcp_runtime.json`. Aider
1076
+ launches with `--mcp-config /tmp/mcp_runtime.json`. **Secrets are never
1077
+ committed.**
1078
+
1079
+ ---
1080
+
1081
+ ## 17. Test Suite β€” What Each Test Asserts
1082
+ <a id="17-test-suite"></a>
1083
+
1084
+ `tests/` contains 9 test modules and one `conftest.py`.
1085
+
1086
+ | Test file | What it asserts |
1087
+ |---|---|
1088
+ | `test_audit_chain.py` | The SHA-256 audit chain in `audit_logger.py` rejects tampering. Writes synthetic events, mutates one entry, expects `verify_chain_integrity()` to flag the break point. |
1089
+ | `test_job_queue.py` | Idempotency guarantees: identical `(tenant, repo, test_path)` submissions don't duplicate. Stale RUNNING rows older than 1h reset to PENDING. Status transitions are valid. |
1090
+ | `test_mcp_servers_load.py` | The MCP runtime config materializes correctly: all `__INJECTED__` placeholders are resolved or the server is dropped. No secrets leak into the rendered file. |
1091
+ | `test_model_router.py` | The 5-tier router produces deterministic routing for the same input. Tier promotions happen on capability misses. Cost estimates are monotonic with tier. |
1092
+ | `test_mythos_diagnostics.py` | `mythos.diagnostics.run_diagnostics()` exits 0 in healthy state and non-zero when any required Mythos module is missing. |
1093
+ | `test_nightmode_smoke.py` | Architect night mode initializes without external services and refuses to act when no scope source is configured. |
1094
+ | `test_scope_parser.py` | The HackerOne / Bugcrowd / Intigriti scope parsers correctly partition assets into IN_SCOPE / OUT_OF_SCOPE buckets with edge cases (wildcards, port specs, regex policies). |
1095
+ | `test_skill_registry.py` | The 28 skill files in `architect/skills/` are all parseable, declare required metadata, and are discoverable by `find_skills(query)`. |
1096
+ | `test_webhook_hmac.py` | `webhook_server.py` rejects requests with invalid HMAC signatures and accepts valid ones. Constant-time comparison is used (no timing side channel). |
1097
+
1098
+ `conftest.py` defines fixtures for: temporary `/data` overrides, mock
1099
+ GitHub API, mock OpenRouter API, in-memory SQLite for the job queue and
1100
+ training store, and a synthetic failing-test repo factory.
1101
+
1102
+ ---
1103
+
1104
+ ## 18. Data on Disk β€” Files, SQLite Schemas, Audit Chain
1105
+ <a id="18-data-on-disk"></a>
1106
+
1107
+ ```
1108
+ /data/
1109
+ β”œβ”€β”€ job_queue.sqlite # job_queue.py
1110
+ β”œβ”€β”€ training_store.sqlite # training_store.py (fix_attempts, fix_patterns)
1111
+ β”œβ”€β”€ embedding_index.sqlite # embedding_memory.py SQLite backend
1112
+ β”œβ”€β”€ memory_engine.sqlite # legacy v1 pattern memory
1113
+ β”œβ”€β”€ rhodawk_findings.db # MCP sqlite-findings server
1114
+ β”œβ”€β”€ audit_trail.jsonl # SHA-256 chained event log
1115
+ β”œβ”€β”€ harvester_feed.json # ranked candidate repos
1116
+ β”œβ”€β”€ disclosure_vault/
1117
+ β”‚ └── <finding_id>.fernet # encrypted writeups
1118
+ β”œβ”€β”€ lora_exports/
1119
+ β”‚ └── lora_<timestamp>.jsonl # SFT-ready training data
1120
+ β”œβ”€β”€ hermes_sessions/
1121
+ β”‚ └── <session_id>.json # persisted Hermes research sessions
1122
+ β”œβ”€β”€ findings/
1123
+ β”‚ └── <finding_id>.json # raw findings from red team / Mythos
1124
+ β”œβ”€β”€ repo/
1125
+ β”‚ └── <owner>__<name>/ # cloned target repos (ephemeral)
1126
+ β”œβ”€β”€ nuclei-templates/ # nuclei DAST templates cache
1127
+ └── research/ # MCP filesystem-research scratch
1128
+ ```
1129
+
1130
+ `/tmp/`:
1131
+ ```
1132
+ /tmp/
1133
+ β”œβ”€β”€ mcp_runtime.json # rendered MCP config (with secrets)
1134
+ β”œβ”€β”€ sandbox/<id>/ # bwrap sandbox roots
1135
+ └── findings/ # MCP filesystem-research output
1136
+ ```
1137
+
1138
+ ### 18.1 SQLite schemas (effective)
1139
+
1140
+ ```sql
1141
+ -- training_store.sqlite
1142
+ CREATE TABLE fix_attempts (
1143
+ id INTEGER PRIMARY KEY,
1144
+ ts TEXT NOT NULL,
1145
+ repo TEXT NOT NULL,
1146
+ test_path TEXT NOT NULL,
1147
+ failure TEXT NOT NULL,
1148
+ diff TEXT,
1149
+ model TEXT,
1150
+ attempt INTEGER,
1151
+ success INTEGER NOT NULL
1152
+ );
1153
+ CREATE INDEX ix_fix_attempts_repo ON fix_attempts(repo);
1154
+ CREATE INDEX ix_fix_attempts_success ON fix_attempts(success);
1155
+
1156
+ CREATE TABLE fix_patterns (
1157
+ signature TEXT PRIMARY KEY,
1158
+ success_count INTEGER NOT NULL DEFAULT 0,
1159
+ attempt_count INTEGER NOT NULL DEFAULT 0,
1160
+ last_seen TEXT NOT NULL
1161
+ );
1162
+
1163
+ -- job_queue.sqlite
1164
+ CREATE TABLE jobs (
1165
+ tenant_id TEXT NOT NULL,
1166
+ repo TEXT NOT NULL,
1167
+ test_path TEXT NOT NULL,
1168
+ status TEXT NOT NULL,
1169
+ pr_url TEXT,
1170
+ model_version TEXT,
1171
+ started_at TEXT,
1172
+ updated_at TEXT NOT NULL,
1173
+ attempts INTEGER NOT NULL DEFAULT 0,
1174
+ last_error TEXT,
1175
+ PRIMARY KEY (tenant_id, repo, test_path)
1176
+ );
1177
+
1178
+ -- embedding_index.sqlite (sqlite-vec)
1179
+ CREATE TABLE embeddings (
1180
+ id INTEGER PRIMARY KEY,
1181
+ failure_norm TEXT,
1182
+ embedding BLOB, -- vector serialized via sqlite-vec
1183
+ fix_diff TEXT,
1184
+ outcome TEXT,
1185
+ repo TEXT,
1186
+ ts TEXT
1187
+ );
1188
+ CREATE VIRTUAL TABLE embedding_index USING vec0(
1189
+ embedding float[384]
1190
+ );
1191
+ ```
1192
+
1193
+ ### 18.2 Audit chain entry shape
1194
+ ```json
1195
+ {
1196
+ "ts": "2026-04-22T13:14:15.123Z",
1197
+ "event": "ADVERSARIAL_REVIEW",
1198
+ "tenant": "default",
1199
+ "repo": "owner/name",
1200
+ "test_path": "tests/test_x.py",
1201
+ "payload": {
1202
+ "verdict": "APPROVE",
1203
+ "confidence": 0.93,
1204
+ "consensus_fraction": 1.0,
1205
+ "models": ["qwen-2.5-7b", "gemma-2-9b", "mistral-7b"],
1206
+ "critical_issues": [],
1207
+ "warnings": ["minor style nit"]
1208
+ },
1209
+ "prev_hash": "sha256:f3a1...c0",
1210
+ "hash": "sha256:9b22...ee"
1211
+ }
1212
+ ```
1213
+
1214
+ ---
1215
+
1216
+ ## 19. Complete Environment Variable Reference (Updated)
1217
+ <a id="19-env-vars"></a>
1218
+
1219
+ This table supersedes the playbook's reference for everything related to
1220
+ inference providers and the build-changed paths.
1221
+
1222
+ ### 19.1 Required for any meaningful run
1223
+ | Var | Purpose |
1224
+ |---|---|
1225
+ | `GITHUB_TOKEN` | PAT with `repo` scope (or use App mode below) |
1226
+ | `OPENROUTER_API_KEY` | needed for fallback + adversarial trio |
1227
+ | `DO_INFERENCE_API_KEY` | strongly recommended β€” enables paid DO primary |
1228
+
1229
+ ### 19.2 Inference providers (new since playbook)
1230
+ | Var | Default | Description |
1231
+ |---|---|---|
1232
+ | `DO_INFERENCE_API_KEY` | β€” | DigitalOcean Serverless Inference key |
1233
+ | `DO_INFERENCE_BASE_URL` | `https://inference.do-ai.run/v1` | DO endpoint |
1234
+ | `DO_INFERENCE_MODEL` | `llama3.3-70b-instruct` | Aider's primary model on DO |
1235
+ | `HERMES_DO_MODEL` | `${DO_INFERENCE_MODEL}` | Hermes' primary model on DO |
1236
+ | `HERMES_OR_MODEL` | `qwen/qwen-2.5-coder-32b-instruct:free` | Hermes' OpenRouter fallback |
1237
+ | `OPENROUTER_API_KEY` | β€” | Fallback + adversarial models |
1238
+
1239
+ ### 19.3 Models, retries, consensus
1240
+ | Var | Default | Description |
1241
+ |---|---|---|
1242
+ | `RHODAWK_MODEL` | (computed: DO if available else OR free) | Aider primary model |
1243
+ | `RHODAWK_ADVERSARY_MODEL` | `openrouter/qwen/qwen-2.5-7b-instruct:free` | lead reviewer |
1244
+ | `RHODAWK_CONSENSUS_THRESHOLD` | `0.67` | majority fraction |
1245
+ | `RHODAWK_ADVERSARY_SEQUENTIAL` | `false` | serial vs concurrent reviewers |
1246
+ | `MAX_RETRIES` | `5` | healing-loop retry budget |
1247
+ | `ADVERSARIAL_REJECTION_MULTIPLIER` | `2` | extra retries when reject reason is adversarial |
1248
+
1249
+ ### 19.4 Conviction (auto-merge)
1250
+ | Var | Default | Description |
1251
+ |---|---|---|
1252
+ | `RHODAWK_AUTO_MERGE` | `false` | enable autonomous merge |
1253
+ | `RHODAWK_CONVICTION_CONFIDENCE` | `0.92` | min adversarial confidence |
1254
+ | `RHODAWK_CONVICTION_CONSENSUS` | `0.85` | min consensus fraction |
1255
+ | `RHODAWK_CONVICTION_MEMORY_SIM` | `0.85` | min memory similarity |
1256
+
1257
+ ### 19.5 Memory backend
1258
+ | Var | Default | Description |
1259
+ |---|---|---|
1260
+ | `RHODAWK_EMBEDDING_BACKEND` | `sqlite` | `sqlite` or `qdrant` |
1261
+ | `RHODAWK_EMBEDDING_MODEL` | `all-MiniLM-L6-v2` | SQLite backend encoder |
1262
+ | `RHODAWK_CODEBERT_MODEL` | `microsoft/codebert-base` | Qdrant backend encoder |
1263
+ | `QDRANT_URL` | β€” | Qdrant remote URL when not local |
1264
+ | `QDRANT_API_KEY` | β€” | Qdrant cloud auth |
1265
+
1266
+ ### 19.6 LoRA scheduler
1267
+ | Var | Default | Description |
1268
+ |---|---|---|
1269
+ | `RHODAWK_LORA_ENABLED` | `false` | enable scheduler |
1270
+ | `RHODAWK_LORA_MIN_SAMPLES` | `50` | min new fixes per export |
1271
+ | `RHODAWK_LORA_MAX_AGE_HOURS` | `168` | max time between exports |
1272
+ | `RHODAWK_LORA_OUTPUT_DIR` | `/data/lora_exports` | JSONL destination |
1273
+
1274
+ ### 19.7 Harvester
1275
+ | Var | Default | Description |
1276
+ |---|---|---|
1277
+ | `RHODAWK_HARVESTER_ENABLED` | `false` | start daemon |
1278
+ | `RHODAWK_HARVESTER_POLL_SECONDS` | `21600` | cycle interval |
1279
+ | `RHODAWK_HARVESTER_MIN_STARS` | `100` | min stars to consider |
1280
+ | `RHODAWK_HARVESTER_MAX_REPOS` | `20` | targets per cycle |
1281
+
1282
+ ### 19.8 Worker pool & isolation
1283
+ | Var | Default | Description |
1284
+ |---|---|---|
1285
+ | `RHODAWK_WORKERS` | `8` | parallel workers |
1286
+ | `RHODAWK_PROCESS_ISOLATE` | `false` | per-job subprocess isolation |
1287
+ | `RHODAWK_ISOLATE_TIMEOUT` | `600` | per-job timeout (seconds) |
1288
+
1289
+ ### 19.9 GitHub
1290
+ | Var | Default | Description |
1291
+ |---|---|---|
1292
+ | `GITHUB_TOKEN` | β€” | PAT auth |
1293
+ | `RHODAWK_FORK_MODE` | `false` | enable fork-and-PR for any public repo |
1294
+ | `RHODAWK_FORK_OWNER` | _(authenticated user)_ | org to fork into |
1295
+ | `RHODAWK_APP_ID` | β€” | GitHub App ID |
1296
+ | `RHODAWK_APP_PRIVATE_KEY` | β€” | GitHub App PEM |
1297
+ | `RHODAWK_WEBHOOK_SECRET` | β€” | HMAC for `/webhook/github` |
1298
+
1299
+ ### 19.10 Adversarial / red team / formal
1300
+ | Var | Default | Description |
1301
+ |---|---|---|
1302
+ | `RHODAWK_RED_TEAM_ENABLED` | `true` | enable CEGIS fuzzer when all tests green |
1303
+ | `RHODAWK_Z3_ENABLED` | `false` | enable Z3 formal verification gate |
1304
+
1305
+ ### 19.11 Architect / Mythos / night mode
1306
+ | Var | Default | Description |
1307
+ |---|---|---|
1308
+ | `ARCHITECT_NIGHTMODE_ENABLED` | `false` | enable autonomous night mode |
1309
+ | `ARCHITECT_GODMODE_REQUIRED` | `true` | require unanimous consensus before external action |
1310
+ | `MYTHOS_API_ENABLED` | `false` | start FastAPI on 7863 |
1311
+ | `MYTHOS_API_TOKEN` | β€” | bearer token for the FastAPI |
1312
+ | `DISCLOSURE_VAULT_PASSPHRASE` | β€” | Fernet key derivation passphrase |
1313
+ | `RHODAWK_BOUNTY_AUTO_SUBMIT` | `false` | auto-submit to bounty platforms |
1314
+
1315
+ ### 19.12 External intelligence
1316
+ | Var | Description |
1317
+ |---|---|
1318
+ | `BRAVE_API_KEY` | Brave Search MCP |
1319
+ | `SHODAN_API_KEY` | Shodan recon MCP |
1320
+ | `NVD_API_KEY` | NVD CVE intelligence MCP |
1321
+ | `NUCLEI_API_KEY` | Nuclei templates premium feed |
1322
+ | `SEMGREP_APP_TOKEN` | Semgrep Cloud rules |
1323
+ | `HACKERONE_USERNAME`, `HACKERONE_API_TOKEN`, `HACKERONE_API_KEY` | HackerOne |
1324
+ | `BUGCROWD_API_TOKEN` | Bugcrowd |
1325
+ | `INTIGRITI_API_TOKEN` | Intigriti |
1326
+
1327
+ ### 19.13 Notifications
1328
+ | Var | Description |
1329
+ |---|---|
1330
+ | `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | Telegram alerts |
1331
+ | `SLACK_WEBHOOK_URL` | Slack alerts |
1332
+
1333
+ ### 19.14 Multitenancy
1334
+ | Var | Default | Description |
1335
+ |---|---|---|
1336
+ | `RHODAWK_TENANT_ID` | `default` | namespace prefix for jobs, audits, exports |
1337
+
1338
+ ---
1339
+
1340
+ ## 20. Failure Modes and Recovery Procedures
1341
+ <a id="20-failure-modes"></a>
1342
+
1343
+ | Symptom | Most likely cause | Recovery |
1344
+ |---|---|---|
1345
+ | Build fails on `pillow` resolver conflict | `requirements.txt` re-lists `aider-chat` | Remove it; aider must be `--no-deps` only in Dockerfile |
1346
+ | Build fails with `gradio[oauth,mcp]==5.29.0` injection | Space SDK reverted to `gradio` | Confirm `sdk: docker` in `README.md` front-matter |
1347
+ | Aider crashes `litellm has no attribute APIConnectionError` | litellm 1.75.0 reinstalled by transitive dep | Re-run the `pip install --no-deps litellm==1.78.5` step or pin in Dockerfile |
1348
+ | All LLM calls fail with 429 | OpenRouter rate-limited and DO not configured | Set `DO_INFERENCE_API_KEY`; calls will route to DO first |
1349
+ | All LLM calls fail with `No inference provider configured` | Both `DO_INFERENCE_API_KEY` and `OPENROUTER_API_KEY` missing | Set at least one; restart container |
1350
+ | Webhook 401 from GitHub | `RHODAWK_WEBHOOK_SECRET` mismatch | Re-set the secret in the GitHub repo's webhook config to match |
1351
+ | Audit hangs forever on a single test | Aider subprocess wedged | Set `RHODAWK_PROCESS_ISOLATE=true` and `RHODAWK_ISOLATE_TIMEOUT=600` |
1352
+ | `embedding_index.sqlite` corrupted | Hard kill mid-write | Delete the file; `rebuild_embedding_index()` will reconstruct from `training_store.sqlite` |
1353
+ | Audit chain integrity fails | Manual edit of `audit_trail.jsonl` | The chain is correctly rejecting tamper. To start a new chain, archive and remove the file; a new genesis entry will be written. |
1354
+ | Conviction never fires | Memory has zero matches at threshold β‰₯0.85 | Expected on cold start; lower `RHODAWK_CONVICTION_MEMORY_SIM` to 0.75 in early life if you want auto-merge sooner β€” but be honest with yourself about what you're trading. |
1355
+ | MCP server fails to spawn | `npx`/`uvx` missing or required env var unset | Check container has nodejs+npm and uvx (uv); confirm injected env var is present at startup |
1356
+ | Harvester feed empty | Token lacks search scope or rate limited | Confirm `GITHUB_TOKEN` has `public_repo`; raise `RHODAWK_HARVESTER_MIN_STARS` |
1357
+ | Mythos FastAPI 401 | Wrong bearer token | Confirm `MYTHOS_API_TOKEN` matches client header |
1358
+
1359
+ ---
1360
+
1361
+ ## 21. Security Model β€” Trust Boundaries and Threat Mitigation
1362
+ <a id="21-security-model"></a>
1363
+
1364
+ ### 21.1 Trust boundaries
1365
+ 1. **Container ↔ host.** Container runs as root in the Docker image but
1366
+ touches only `/data` (mounted) and `/tmp` (ephemeral). No host paths
1367
+ leak.
1368
+ 2. **Container ↔ target repo.** Cloned repos live in `/data/repo/<id>/`.
1369
+ They are treated as untrusted: scripts in target repos are never
1370
+ `python <target_script>`'d directly β€” only the runtime's allowed
1371
+ commands (`pytest`, `npm test`, `go test`, …) are invoked.
1372
+ 3. **Container ↔ LLM provider.** Code snippets and failure traces are
1373
+ sent over the wire to DigitalOcean Inference and OpenRouter. This is
1374
+ the primary data-leakage surface β€” controlled by deployment mode (HF
1375
+ Spaces is multi-tenant; self-hosted is single-tenant).
1376
+ 4. **MCP fetch ↔ internet.** `fetch-docs` and similar fetch MCPs use
1377
+ `FETCH_ALLOWED_DOMAINS` allow-lists to prevent SSRF against internal
1378
+ services.
1379
+ 5. **Webhook ↔ public internet.** All webhook endpoints validate HMAC
1380
+ signatures with constant-time comparison. Per-IP rate limiting in
1381
+ front. No untrusted JSON is unpickled.
1382
+
1383
+ ### 21.2 Threat mitigation by attack
1384
+ | Threat | Mitigation |
1385
+ |---|---|
1386
+ | Prompt injection in target source | Multi-model adversarial review catches divergent reactions; SAST/supply-chain catch the resulting bad diffs |
1387
+ | LLM-introduced typosquat (`reqeusts`) | `supply_chain.py` Levenshtein typosquatting check |
1388
+ | LLM-introduced backdoor | SAST + adversarial trio + zero-package-introduction conviction criterion |
1389
+ | Tampered audit trail | SHA-256 chain detects any post-hoc edit |
1390
+ | Webhook spoofing | HMAC-SHA256 with constant-time compare |
1391
+ | Secret leakage via logs | Secret scanner runs over diffs; `mcp_runtime.json` is in `/tmp` not the repo |
1392
+ | Subprocess sandbox escape | Runtime command allow-lists; optional bwrap; planned Firecracker |
1393
+ | Bug-bounty submission of duplicate finding | `cve_intel.py` correlation + manual approval default |
1394
+ | Auto-merge of an unsafe diff | 7 conviction criteria; default OFF; passes only the cleanest fixes |
1395
+ | Provider compromise (e.g., OpenRouter MITM) | TLS verification; provider chain falls back rather than blindly trusting bad responses |
1396
+
1397
+ ### 21.3 Secret hygiene
1398
+ - `mcp_config.json` is a **template** with `__INJECTED_BY_APP_AT_RUNTIME__`
1399
+ placeholders. The realized config goes to `/tmp/mcp_runtime.json` which
1400
+ is not committed and is recreated on each startup.
1401
+ - `.env`-style files are never read; everything goes through
1402
+ `os.getenv`. HF Spaces / Docker `-e` are the only injection paths.
1403
+ - The disclosure vault uses Fernet with key derived from
1404
+ `DISCLOSURE_VAULT_PASSPHRASE`. Lose that passphrase, lose the vault β€”
1405
+ by design.
1406
+
1407
+ ---
1408
+
1409
+ ## 22. Migration Playbook β€” HF Space β†’ Paid Server
1410
+ <a id="22-migration"></a>
1411
+
1412
+ The Space is a great PoC environment but has hard limits: ephemeral
1413
+ storage on free tier, single-container topology, and shared GPU resources.
1414
+ The paid-server migration target is a single Linux box (DigitalOcean
1415
+ Droplet, Hetzner CCX, or AWS EC2) running the same Docker image.
1416
+
1417
+ ### 22.1 Pre-migration checklist
1418
+ - [ ] All env vars from Β§19 documented and stored in a secrets manager
1419
+ - [ ] `/data` directory size on Space measured (`du -sh /data`) and
1420
+ provisioned on the new host (recommend β‰₯50 GB to start)
1421
+ - [ ] DigitalOcean Inference key budgeted for the expected request volume
1422
+ - [ ] Webhook secret rotated for the new endpoint URL
1423
+ - [ ] DNS or reverse-proxy plan in place if you want HTTPS in front of
1424
+ ports 7860/7861/7862/7863
1425
+ - [ ] Decision made on Qdrant: keep SQLite/MiniLM (simpler) or stand up
1426
+ a Qdrant container alongside (better recall on large corpora)
1427
+
1428
+ ### 22.2 Migration steps
1429
+ 1. **Snapshot data.** On the Space:
1430
+ ```bash
1431
+ tar czf /tmp/rhodawk-data.tgz /data
1432
+ ```
1433
+ Download via the HF Spaces file browser or via git LFS.
1434
+ 2. **Provision the new host.** Install Docker + Docker Compose. Mount a
1435
+ block volume at `/var/lib/rhodawk-data`.
1436
+ 3. **Restore data.**
1437
+ ```bash
1438
+ sudo mkdir -p /var/lib/rhodawk-data
1439
+ sudo tar xzf rhodawk-data.tgz -C /var/lib/rhodawk-data --strip-components=1
1440
+ ```
1441
+ 4. **Build the image** (or pull from a private registry if you've pushed
1442
+ one):
1443
+ ```bash
1444
+ git clone https://huggingface.co/spaces/Architect8999/rhodawk-ai-devops-engine
1445
+ cd rhodawk-ai-devops-engine
1446
+ docker build -t rhodawk:latest .
1447
+ ```
1448
+ 5. **Run with all env vars + persistent volume.**
1449
+ ```bash
1450
+ docker run -d --name rhodawk \
1451
+ -p 7860:7860 -p 7861:7861 \
1452
+ -v /var/lib/rhodawk-data:/data \
1453
+ -e DO_INFERENCE_API_KEY=... \
1454
+ -e OPENROUTER_API_KEY=... \
1455
+ -e GITHUB_TOKEN=... \
1456
+ -e RHODAWK_WEBHOOK_SECRET=... \
1457
+ -e RHODAWK_TENANT_ID=acme \
1458
+ -e RHODAWK_AUTO_MERGE=false \
1459
+ -e RHODAWK_HARVESTER_ENABLED=true \
1460
+ rhodawk:latest
1461
+ ```
1462
+ 6. **Update GitHub webhook** payload URL to the new host.
1463
+ 7. **Verify the audit chain integrity** on the new host:
1464
+ ```bash
1465
+ docker exec rhodawk python -c \
1466
+ "from audit_logger import verify_chain_integrity; verify_chain_integrity()"
1467
+ ```
1468
+ 8. **Run a known-failing-test repo** through the chat inbox or a webhook
1469
+ to confirm end-to-end.
1470
+
1471
+ ### 22.3 Recommended additions on paid host
1472
+ - Reverse proxy (Caddy or Traefik) terminating TLS, fronting 7860/7861.
1473
+ - Docker Compose file checking the container into a restart-policy.
1474
+ - `cron` or systemd timer that runs `docker exec rhodawk python -m
1475
+ mythos.diagnostics` every 5 minutes and pages on non-zero exit.
1476
+ - Off-host backup of `/var/lib/rhodawk-data` nightly. The training store
1477
+ is the proprietary asset β€” losing it loses the moat.
1478
+ - Optional Qdrant container (`qdrant/qdrant:latest`) on a private network,
1479
+ with `RHODAWK_EMBEDDING_BACKEND=qdrant` + `QDRANT_URL=http://qdrant:6333`.
1480
+
1481
+ ### 22.4 What changes at scale (later)
1482
+ - Move `job_queue` from SQLite to PostgreSQL (`psycopg2-binary` already
1483
+ installed, see `mcp_config.json` postgres-intelligence server).
1484
+ - Move audit trail to S3 with versioning (the SHA-256 chain remains; only
1485
+ the storage backend changes).
1486
+ - Replace single-host worker pool with a queue + worker fleet
1487
+ (Redis/RQ or Celery).
1488
+ - Replace Gradio dashboard with a Next.js front-end calling the
1489
+ `mythos.api.fastapi_server` JSON API.
1490
+
1491
+ ---
1492
+
1493
+ ## 23. Glossary of Internal Names
1494
+ <a id="23-glossary"></a>
1495
+
1496
+ | Name | Meaning |
1497
+ |---|---|
1498
+ | **Aider** | Open-source AI pair-programming CLI, used as the patch generator |
1499
+ | **APIConnectionError** | Symbol that aider expects in `litellm`; missing in 1.75.0 β€” fixed by upgrading to 1.78.5 |
1500
+ | **ACTS** | Adversarial Consensus Trust Score (Hermes) |
1501
+ | **Audit chain** | The SHA-256-chained JSONL log in `audit_logger.py` |
1502
+ | **Architect** | Higher-order control package (`architect/`) β€” tier router, sandbox, skills, night mode |
1503
+ | **CEGIS** | Counterexample-Guided Inductive Synthesis (red team loop) |
1504
+ | **Conviction** | The 7-criteria gate that allows autonomous merge |
1505
+ | **DO** | DigitalOcean Serverless Inference (the new primary inference provider) |
1506
+ | **EmbodiedOS** | External persistent runtime that hosts OpenClaw + Hermes Agent (optional bridge) |
1507
+ | **Fork mode** | Forking any public repo and PR'ing back β€” enables fixing repos you don't own |
1508
+ | **Godmode consensus** | Stricter consensus (adversarial trio + Tier 1) used by night mode |
1509
+ | **Harvester** | Daemon that finds its own targets on GitHub |
1510
+ | **Hermes** | The 6-phase research orchestrator (`hermes_orchestrator.py`) |
1511
+ | **LoRA scheduler** | Daemon that exports successful (failure, fix) pairs as SFT JSONL |
1512
+ | **MCP** | Model Context Protocol β€” Anthropic's tool-server protocol |
1513
+ | **Mythos** | The deepest research package (`mythos/`) β€” multi-agent + RL + 17 native MCP servers |
1514
+ | **Mythos-level** | Aspirational target capability class β€” autonomous frontier vulnerability research |
1515
+ | **Night mode** | Autonomous bug-bounty loop running between human-driven audits |
1516
+ | **OpenClaw** | Local-gateway component of EmbodiedOS |
1517
+ | **OR** | OpenRouter (the new fallback inference provider) |
1518
+ | **PoC** | Proof of Concept (exploit) |
1519
+ | **Provider chain** | The ordered list `[DO_primary, OpenRouter_fallback]` consulted on every LLM call |
1520
+ | **Sandbox** | bwrap (or planned Firecracker) isolation for tool execution |
1521
+ | **SAST gate** | Static analysis security gate (bandit + semgrep + secret patterns) |
1522
+ | **Supply chain gate** | CVE + typosquatting check on newly added packages |
1523
+ | **Tier 1–5** | Five-tier model router classification |
1524
+ | **Training store** | SQLite DB of every fix attempt β€” the proprietary data asset |
1525
+ | **TVG** | Threat-Vector Graph |
1526
+ | **VES** | Vulnerability Evidence Score (Hermes) |
1527
+ | **Z3 gate** | Optional formal-verification gate using the Z3 SMT solver |
1528
+
1529
+ ---
1530
+
1531
+ *Rhodawk AI β€” System Analysis Book*
1532
+ *Companion to `FOUNDER_PLAYBOOK.md` v4.0 (which remains the canonical investor narrative)*
1533
+ *Generated from the live source tree; last updated to reflect the DigitalOcean primary / OpenRouter fallback inference chain, the `--no-deps` aider install pattern, and the Docker-SDK Space configuration.*