docs/features/opencoti_server.md

F3 — opencoti-server (Go companion daemon)

Parent: ../MASTER_PLAN.md Status: planning Owner: TBD Reference: /shared/dev/claude-hooks (the inspiration; mirror its goals, then go further)

Problem

claude-hooks is a Python daemon that wraps Claude Code with a hook API and adds persistent memory, multi-session coordination, tool glue, and a small zoo of integrations. It works very well for Claude Code. opencoti has the same shape of needs — and a few extra because opencoti has its own tier engine, its own local llamafile, and its own session model.

We want a Go daemon that is to opencoti what claude-hooks is to Claude Code, but deeper because we control opencoti's source. Instead of bolting onto a hook API from outside, opencoti-server can be a first-class peer of opencoti's runtime.

Goals

• G1. Single Go binary, opencoti-server, that exposes a small, well-versioned API to opencoti.
• G2. Persistent memory (vector + KG) shared across opencoti sessions on a host.
• G3. Multi-session coordination: opencoti CLI instances on the same host see each other, share context where the user wants it.
• G4. Tool glue: a place to register external tools (search, filesystem extensions, MCP-like) once and have them appear in all opencoti sessions.
• G5. Event sink for tier-engine routing decisions (so we can log, audit, and later learn from them).
• G6. Hook fan-out compatible with the existing claude-hooks ecosystem where it makes sense — but not bound by it.

Non-goals (for now)

• Replacing claude-hooks. opencoti-server is opencoti-specific.
• Becoming a generic "MCP host". MCP integration is a feature, not the architecture.
• Multi-host federation. Single-host first.

Design sketch

Where it lives

• opencoti/server/ — top-level non-bun source root. Go module. Putting it outside packages/ keeps the bun workspace clean.
• packages/opencoti-server-client/ — TypeScript client used by opencoti to talk to the server.
• Surgical hooks in opencode session lifecycle (session start, session end, turn complete, tool invocation, model call result): each hook is a one-line "if a client is available, notify it". Listed in docs/protocols/UPSTREAM_SYNC.md.

Transport

Unix domain socket on Linux/macOS, named pipe on Windows. Loopback TCP as a fallback. HTTP/JSON wire format for simplicity (no gRPC build dependency for the client).

Daemon-launch policy (applies to opencoti-server and every opencoti daemon):

• Listen port MUST come from opencoti's reserved 47000-48000 range — never 38000-39000 or 18790-18811 (claude-hooks territory). Tentative reservation table to keep adjacent ports clear of accidental collision:
  • 47092 — @opencoti/embedder (F4)
  • 47190 — opencoti-server daemon (proposed default; revisit when F3 M1 ships)
  • 47191 — opencoti-server dashboard (proposed default)
• First-run setup MUST ask the user explicitly where the daemon should bind: 127.0.0.1 (default), all interfaces, or a specific IP. The default port from the reservation table is proposed; the user may override either field within the 47000-48000 range. Setup writes the validated choice to the user's config.

Persistence

Two backends, picked per-deployment:

• pgvector — same as solidPC's claude-hooks setup. Best when a Postgres is already on the host.
• sqlite-vec — single-file, no external dependency. Default for fresh installs.

The backend choice is config; the memory API is identical.

Surface (illustrative)

POST /v1/sessions                          # register a session
POST /v1/sessions/:id/turns                # record a turn
POST /v1/memory                            # store a memory (M2)
POST /v1/memory/search                     # vector recall (M2 — POST because the embedding is multi-KB)
POST /v1/memory/collections                # collections CRUD (M2)
PUT  /v1/memory/collections/:name/acl      # per-session ACL (M2)
GET  /v1/memory/count                      # count (M2)
POST /v1/memory/kg/entities                # KG ops
POST /v1/memory/kg/relations
POST /v1/tier-events                       # log a tier-engine decision
GET  /v1/tools                             # list registered tools
POST /v1/tools/:name/invoke                # invoke (server-side glue)
GET  /v1/healthz                           # M1; M2 extended with store_ok/store_path/embedding_dim/store_error

The server is embedder-agnostic at M2: clients pass vectors, not text. The text→vector convenience (e.g., GET /v1/memory?q=<text> proxying to the embedder daemon) is a deferred follow-up — adding it requires a runtime dep on @opencoti/embedder (F4 M5) and is most naturally landed once the TS client (M3) demonstrates the convenience is wanted at the HTTP layer vs done client-side.

API is versioned (/v1) and the wire schema is owned by packages/opencoti-server-client/ (so opencoti can rev independently of the server binary, within compatible versions).

Mirroring claude-hooks goals

Same goals, mapped:

claude-hooks	opencoti-server
claude-hooks recall hooks	implicit recall on /v1/memory query, surfaced by opencoti's prompt builder
mcp__pgvector__* MCP tools	/v1/memory (and a thin MCP shim if external MCP clients want it)
Stop hook auto-ingest	server records turn-complete events, applies same heuristics
claude-hooks companion tools (Episodic etc.)	first-class tools registered via /v1/tools

Deeper integration than claude-hooks

Because we own opencoti, we can:

• Receive tier-engine decisions directly (provider chosen, escalation reasons, cost) without scraping logs.
• Receive diff and tool-output events as structured payloads, not parsed from a transcript.
• Push back: server can suggest a memory recall payload that opencoti injects directly into the prompt builder, instead of appending to a transcript.

Milestones

M1 — Server skeleton + healthz (2026-05-23 — shipped)

• Go module under opencoti/server/ (module github.com/mann1x/opencoti/server, Go 1.22). Single dep: golang.org/x/sys for the Windows svc packages.
• opencoti-server serve binds a UDS by default on Unix ($XDG_RUNTIME_DIR/opencoti/server.sock or ~/.opencoti/server.sock fallback) or tcp://127.0.0.1:47190 on Windows. --addr unix:///path and --addr tcp://host:port override.
• One HTTP endpoint: GET /v1/healthz returns 200 + JSON {status, version, started_at, uptime_seconds}. All other paths return a 404 envelope with code + message so the wire format is consistent for M2+. POST /v1/healthz returns 405 with the same envelope.
• Graceful shutdown on SIGTERM/SIGINT (Unix) or SCM Stop/Shutdown (Windows). UDS file unlinked on Unix shutdown.
• Windows service support shipped in M1 (per user directive 2026-05-23):
  • install subcommand wraps SCM (golang.org/x/sys/windows/svc/mgr): --name, --display-name, --description, --start-type=auto|manual|disabled, --addr. Registers the Event Log source under the same name.
  • uninstall removes the SCM entry AND the Event Log source.
  • start / stop wrap mgr.Service.Start / Service.Control(svc.Stop). stop waits up to 10s for the state to become Stopped.
  • When svc.IsWindowsService() returns true, serve enters svc.Run with a handler that translates SCM Stop / Shutdown into a graceful http.Server.Shutdown. Log records flow to the Windows Application Event Log via eventlog.Open(name).
  • Single source tree, build-tagged platform splits (*_windows.go / *_nonwindows.go). Same binary surface on every platform — install / uninstall / start / stop on Unix print "Windows-only command" and exit 2.
• 7 cross-platform tests (parser + listener + healthz/404/405
  • UDS roundtrip + UDS unlink-on-cleanup + double-shutdown safety) + 3 Windows-tagged tests for the BuildMgrConfig install-config builder. All green on Linux; Windows cross-compile clean.
• Hook footprint: ZERO. M1 is purely additive — no opencoti-hook: markers, no UPSTREAM_SYNC.md registry rows. The autostart hook lands in M3.
• Deferred from M1: Windows named-pipe TRANSPORT (use TCP on Windows in M1; named-pipe support is a follow-up); systemd/launchd unit files on Unix; CI workflows (the repo has no CI yet — separate concern); the first-run setup wizard step ("install as Windows service?" — comes when the setup-flow picks up F3 concerns).

M2 — Memory backend (sqlite-vec) + recall API (2026-05-23 — shipped)

• Go SQLite stack: github.com/mattn/go-sqlite3 (cgo) + github.com/asg017/sqlite-vec-go-bindings/cgo, sqlite-vec statically linked via sqlite_vec.Auto() so the daemon ships as a single binary (no vec0.so to bundle). CGO becomes a hard build requirement at M2; cross-compile to Windows from Linux needs MinGW-w64 (apt install gcc-mingw-w64-x86-64).
• internal/store/store.go defines a Store interface mirroring the TS MemoryStore in packages/opencoti-memory/src/types.ts 1:1. M7's pgvector backend will implement the same interface, so the HTTP layer doesn't change.
• internal/store/sqlitevec/ is the M2 backend. DDL mirrors packages/opencoti-memory/src/schema.ts byte-functionally: four tables (meta, collections, memories, session_acl) plus the memory_vecs vec0 virtual table; SCHEMA_VERSION=1; embedding_dim parameter-substituted into the vec0 DDL at first creation and stored in meta for validation on later opens. Content idempotency: SHA256-hex with UNIQUE(collection_name, content_hash). Pure ResolveAccessMode mirrors the TS resolver (global=r, session-owner=rw, others=none; explicit session_acl rows override). Recall is vec0 MATCH + overscan (k * min(8, max(2, len(candidates)))) + join-back + ACL filter, truncated to k.
• DB schema is wire-compatible with the in-process @opencoti/memory package — a DB created by either side opens cleanly under the other. This is the load-bearing invariant for M3's path-B refactor.
• HTTP surface (eight new routes, all under /v1/memory/*):
  • POST /v1/memory/collections — create (201 / 409 collection_exists / 400 invalid_collection_name).
  • GET /v1/memory/collections?session_id=... — list, optionally filtered through the ACL resolver.
  • DELETE /v1/memory/collections/{name} — cascade-deletes memories + vec0 rows + session_acl entries.
  • PUT /v1/memory/collections/{name}/acl — set per-session ACL (r/w/rw/none).
  • GET /v1/memory/collections/{name}/acl?session_id=... — read the resolved access mode.
  • POST /v1/memory — store. 422 dim_mismatch on length mismatch; 403 write_denied on ACL deny.
  • POST /v1/memory/search — vector recall (POST because a 1024-element Float32 array doesn't fit a query string).
  • GET /v1/memory/count?collection=... — count, optionally scoped to one collection.
• All error responses use M1's {error: {code, message}} envelope. Embeddings on the wire are JSON [float32, ...] arrays (verbose but trivial for the M3 TS client; base64 raw bytes is a follow-up if wire size measurably matters).
• /v1/healthz extended with store_ok, store_path, embedding_dim, and store_error (omitempty). When the store fails to open, the daemon still serves /v1/healthz with store_ok=false and /v1/memory/* returns 503 store_unavailable — partial degradation beats refuse-to-start so the daemon stays observable when something's wrong with the DB.
• serve gets --db-path (default $HOME/.opencoti/memory/state.db — matches the TS defaultDbPath() exactly so both implementations point at the same file by default) and --embedding-dim (default 1024, matches DEFAULT_EMBEDDING_DIM). install bakes both into the SCM service-start arguments.
• 30 tests cover the surface: 18 unit tests on the store conformance (collection lifecycle, ACL resolver, idempotent store, recall ordering, count, close+reopen, dim-mismatch paths) + 12 HTTP integration tests via httptest.NewServer (every endpoint, happy + 4xx + 5xx envelopes).
• testdata/crosslang/ ships a Go↔Bun schema-compat probe: Go writes a DB → optionally invokes a Bun script that opens the same DB via @opencoti/memory and confirms readback. Skips cleanly when bun or the workspace package isn't on PATH. make test-cross-lang is always safe to run.
• Hook footprint: ZERO (unchanged from M1). M2 is purely additive — the API exists but no surgical hooks into opencode yet.
• Deferred from M2: text→vector convenience (GET /v1/memory?q=<text>); streaming recall (SSE); pagination on list-collections; authentication for TCP transport (UDS owner-only on Unix is the M2 security boundary); pgvector backend (M7); migration tools (schemas are identical — the DB just opens); server-side embedder cache; bulk-store / batch endpoints; base64 raw-byte embedding wire format.

M3 — TS client + opencoti hook to start the server (2026-05-23 — shipped)

• @opencoti/opencoti-server-client published in-workspace under packages/opencoti-server-client/. Single class OpencotiServerClient implements MemoryStore over HTTP — all eight /v1/memory/* routes from M2 plus a healthz() probe. Wire format is M2's verbatim: JSON [float32, ...] for embeddings; M1/M2's {error: {code, message}} envelope decoded into a sentinel OpencotiServerError({status, code, message}) so callers can match on code === "store_unavailable", dim_mismatch, etc. Per-request AbortController + 5 s default timeout. close() is a documented no-op (no per-instance handle to close).
• Transport uses Bun's native fetch — tcp://host:port is rewritten to http://... in toFetchBase; unix:///path passes through verbatim (Bun supports UDS fetch natively, no custom adapter needed). defaultAddress(platform) mirrors the Go DefaultAddress() exactly: $XDG_RUNTIME_DIR/opencoti/server.sock on Linux ($HOME/.opencoti/server.sock fallback), and tcp://127.0.0.1:47190 on Windows.
• Autostart lives in src/autostart.ts: startAutostart({address, binaryPath?, dbPath?, embeddingDim?, ...}) probes /v1/healthz first (200 → already_running), locates the binary via Bun.which if none was passed (→ no_binary when missing), spawns with stdio: "ignore" (mirroring the @opencoti/embedder manager pattern), and polls /v1/healthz every 200 ms until the deadline (default 5 s; → ready_timeout if never recovers). Returns a discriminated AutostartOutcome so consumers can log five distinct states (already_running | spawned | no_binary | spawn_failed | ready_timeout) without exception handling. Test seams: spawnImpl, fetchImpl, whichImpl, sleepImpl.
• Surgical hook footprint: 3 markers + 1 package.json dep. All three TS markers land in packages/opencode/src/config/config.ts (the same file that already carries opencoti-default-plugins): the import next to it, the opencoti.server.* schema struct (autostart: boolean, address: string, binary_path: string, db_path: string, embedding_dim: PositiveInt), and the fire-and-forget void maybeStartOpencotiServer(...) call site right after the existing applyDefaultPlugins(...) call. The hook's full substance lives in @opencoti/opencoti-server-client/autostart-hook so the opencode-side surface stays minimal — three lines plus the package.json dep (no anchor comment in JSON, registered in the table).
• Plugin swap. Both @opencoti/memory-plugin and @opencoti/tui-memory accept a new optional server_address?: string. The default StoreFactory: if serverAddress is set, instantiate OpencotiServerClient, call healthz(), and return the client when storeOk === true; otherwise silently fall through to the existing in-process SqliteVecStore.open(...). The __setStoreFactory test seam is unchanged — existing tests continue to inject stubs that ignore serverAddress.
• Health envelope mismatch detection. Client validates that embedding_dim from the server matches the configured dim; on mismatch, healthz() returns storeOk: false with a dim mismatch error so the plugin falls back to the in-process store rather than corrupting the wire format. This is client-side; the server's own dim check still fires on POST /v1/memory.
• Tests. 56 new tests in the client package: 33 mocked-fetch client conformance (every endpoint, happy + 4xx + 5xx envelopes, dedup, dim mismatch, store_unavailable propagation), 11 pure tests for defaultAddress / toFetchBase / joinURL / probeReady, 6 mocked-spawn startAutostart cases (already running, no binary, spawned-then-ready, ready timeout, spawn failed, --addr passthrough). Plus 2 new cases in the memory-plugin tests and 3 new in the tui-memory tests for the server_address swap path. All 23 workspace packages typecheck clean; 82 tests pass across the three affected packages.
• Live verification. Manual end-to-end smoke against the F3 M2 binary (TCP loopback): healthz returns the expected shape (store_ok: true, embedding_dim: 64), createCollection
  • setSessionAcl(rw) + store(...) (with deduplicated: false then true on idempotency) + recall(...) (distance 0 on exact match) + count(...) (returns 1) + deleteCollection all return the expected shapes. UDS path tested at the discover / autostart layer; full UDS smoke deferred to M4 when session-event hooks bring it under day-to-day use.
• Path-B loop closed. With M3 in, the F4 M5 in-process-then-daemon migration path described in memory_embedder.md is mechanically complete: setting opencoti.server.address (or autostart: true) in opencode.jsonc is the only change a user makes to switch from the in-process store to the Go daemon. No code paths change in @opencoti/memory itself.
• Deferred from M3: detached / setsid daemon lifecycle (M3's spawn dies with opencode — cross-session sharing requires running the daemon externally, e.g. the M1 Windows service installer or a future systemd unit); the first-run "install as a service?" wizard step (a follow-up once the setup-flow picks up F3 concerns); a text-q= convenience endpoint (GET /v1/memory?q=<text>) — every M3 caller has the embedder in-process and can pre-embed; retry / backoff on transient HTTP errors (single attempt, the plugin's silent fallback handles the failure case); bulk-store / batch endpoints; TLS / shared-secret authentication for TCP transport (UDS owner-only remains the M3 security boundary).

M4 — Session + turn events flowing (2026-05-23 — shipped)

Design pivot vs the original spec. M4 was originally specced as surgical hooks at session start/end/turn complete. Phase-1 exploration confirmed opencode's @opencode-ai/plugin API already exposes those events fully typed via Hooks.event (session.created/updated/deleted/error/idle plus session.status). Following the precedent set by M5-D2 (@opencoti/memory-plugin), M4 ships as a plugin with zero opencoti-hook footprint — strictly richer than a surgical hook for this use case (typed payloads, zero upstream-source touch, no UPSTREAM_SYNC.md row to maintain on every sync). Hook count stays at 9 source markers + 4 JSON deps (unchanged from F3 M3).

• Per-feature schema versioning. meta.schema_version stays at 1 (memory tables — kept stable so M2/M3 TypeScript @opencoti/memory clients still open M4 DBs cleanly). New meta.sessions_schema_version = 1 is written by M4+ daemons and reported on /v1/healthz. M3 clients without the key treat the daemon as pre-M4; M4 plugins disable forwarding silently when sessionsSchemaVersion < 1.
• Three new tables in the existing state.db (no new --flag, no migration tool):
  • sessions — soft-delete via deleted_at. Indices on parent_id (fork-tree queries) and deleted_at (cheap WHERE deleted_at IS NULL). Upsert is idempotent on id; INSERT ... ON CONFLICT(id) DO UPDATE SET ... overwrites all columns from the latest snapshot.
  • session_events — append-only audit log. FOREIGN KEY ... ON DELETE CASCADE would normally wipe history on DELETE FROM sessions; we use soft-delete instead so the audit trail outlives the session row.
  • session_messages — UPSERT on opencode MessageID. The plugin re-sends the full latest snapshot on every turn boundary; SQLite UPSERT handles dedup.
• SessionStore interface (internal/store/store.go): 8 methods + sentinel errors (ErrSessionNotFound, ErrInvalidSessionID, ErrSessionPayloadTooLarge). The same *sqlitevec.Store implements both Store (memory) and SessionStore; server.go type-asserts at each entry point. M7 pgvector will implement the same interface.
• HTTP surface under /v1/sessions/*:
  • POST /v1/sessions — upsert (201 on insert, 200 on update, 400 invalid_session_id, 413 payload_too_large).
  • GET /v1/sessions[?parent_id=&include_deleted=&limit=&offset=] — list, default newest-first by updated_at, omitting soft-deleted rows.
  • GET /v1/sessions/{id} / DELETE /v1/sessions/{id} — fetch / soft-delete.
  • POST /v1/sessions/{id}/events / GET /v1/sessions/{id}/events[?type=&limit=&offset=] — append + list audit entries.
  • POST /v1/sessions/{id}/messages / GET /v1/sessions/{id}/messages[?limit=&offset=&since=] — bulk-upsert (idempotent on message id) + list.
  • GET /v1/sessions/{id}/turns — coarse server-side projection grouping messages by user-message boundaries. Authoritative turn-boundary logic stays in opencode's compaction.ts; this view is for UI / debugging.
• healthz extension: sessions_schema_version field added; M3 clients without the field treat the daemon as pre-M4.
• Payload size guards: 8 MiB hard cap on session_events.payload and the bulk-upsert messages body (413 payload_too_large); the plugin chunks at a softer 1 MiB default before posting.
• TS client extension (@opencoti/opencoti-server-client): 8 new OpencotiServerClient methods (upsertSession, listSessions, getSession, deleteSession, appendSessionEvent, listSessionEvents, upsertSessionMessages, listSessionMessages, listSessionTurns). Wire format snake_case → camelCase via private mappers (same pattern as M2's mapHit / mapCollectionInfo). healthz() return type + ProbeResult gain sessionsSchemaVersion?: number.
• New plugin @opencoti/session-events-plugin: subscribes to session.created/updated/deleted/error/idle and session.status (idle transition) via opencode's typed Hooks.event; does not subscribe to message.updated (fires thousands of times per turn — text deltas, tool calls, reasoning chunks — and forwarding every delta would melt the daemon). On idle, the plugin fetches the session's latest messages via input.client.session.messages.list and bulk-upserts in byte-bounded chunks. Maintains a per-session merge cache so opencode's partial session.updated payloads are reconciled into a full snapshot before reaching the daemon. Best-effort, never-throws: a forwarding plugin must not propagate daemon failures into the opencode session.
• session.error before sessionID exists lands on a synthetic __pre_session_errors__ session (lazily upserted on first such error) so the audit log captures pre-creation failures without losing context.
• Auto-wire: one-line addition to @opencoti/tiers/default-plugins OPENCOTI_DEFAULT_PLUGINS. Any user with opencoti config in opencode.jsonc gets the plugin automatically; without an opencoti.server.address the plugin's healthz probe fails, the internal client stays undefined, and every event handler early-returns — same no-op-when-unconfigured ergonomic the memory-plugin uses.
• Tests: 26 new Go tests (16 store unit + 10 HTTP integration) — total Go test count climbs from 30 → 63. 14 new TS client tests (33 → 47). 16 new plugin tests in @opencoti/session-events-plugin. All 24 workspace packages typecheck clean.
• Live verification: end-to-end smoke against a real M4 daemon (TCP loopback) confirmed healthz reports sessions_schema_version: 1, full upsert → update → get → list → events → messages → turns → soft-delete cycle works, soft-deleted session's events + messages remain queryable through include_deleted=true.
• Deferred from M4:
  • message.updated real-time forwarding — coalescing on idle is the right granularity; per-delta forwarding would mean O(text-deltas) HTTP roundtrips per turn.
  • Per-tool-invocation events as first-class rows — captured today inside the message's parts JSON; can be promoted to a session_tool_calls table later if needed.
  • WebSocket / SSE push from daemon → client — request-response only in M4.
  • TLS / authentication for /v1/sessions/* TCP transport — UDS owner-only on Unix remains the M4 security boundary.

M5 — Tier-engine event sink (shipped 2026-05-23)

• Daemon side: /v1/tier-events.

  • POST /v1/tier-events appends a row (201; 400 invalid_tier_event; 413 payload_too_large at the same 8 MiB cap as /v1/sessions/*). Body: {event_type, ts?, session_id?, payload} where payload is opaque JSON — the daemon does not introspect it.
  • GET /v1/tier-events?session_id=&event_type=&since=&limit=&offset= returns the audit log newest-first.
  • One new tier_events table: (id PK AUTO, session_id TEXT, event_type TEXT, payload TEXT, ts INTEGER) + two indices (session_id, ts) and (event_type, ts). session_id is a soft reference — no FK, no cascade. Tier events fire intra-turn and can land before session.created reaches the daemon; the soft reference guarantees no event is dropped.
  • Per-feature meta.tier_events_schema_version = 1. The cross-language sqlite_vec contract continues to live on meta.schema_version = 1 (still M2-compatible with the TS @opencoti/memory reader); the new key is independent.

• TS plumbing. @opencoti/opencoti-server-client gains appendTierEvent + listTierEvents (camelCase→snake_case wire mapping mirroring mapSessionMessage). healthz() surfaces tierEventsSchemaVersion?: number; plugins use it as their feature gate.

• Design pivot continues: plugin path > surgical hook. The Telemetry interface in @opencoti/tiers (designed for this milestone — its docstring at telemetry.ts:1-4 said so) is already threaded end-to-end through runtime.ts:99 → executor.ts → escalator.ts → fanout.ts. M5 plugs a real sink into the existing seam via a new module-level registered-telemetry.ts slot (mirrors active-config.ts). Precedence chain in runtime.ts:99 becomes:

  const telemetry =
    hook.telemetry ??
    execOpts.telemetry ??
    getRegisteredTelemetry() ??
    noopTelemetry
  

  The new @opencoti/tier-events-plugin calls setRegisteredTelemetry(impl) during plugin boot when the daemon's healthz reports tierEventsSchemaVersion >= 1. Zero new opencoti-hook source markers — count unchanged from F3 M4 (9 source + 4 JSON deps + 3 prose). See the F1 page for where the Telemetry interface itself was added.

• Fire-and-forget posture. The plugin's Telemetry impl maps each method to a client.appendTierEvent(...).catch(() => {}) — synchronous to the caller, HTTP swallowed on failure. The tier engine never blocks on the daemon. Cf. M4's @opencoti/session-events-plugin, same design.

• Auto-wire via defaults. @opencoti/tier-events-plugin is in OPENCOTI_DEFAULT_PLUGINS alongside the M4 session plugin. Any user with opencoti: {...} in their config gets the audit log for free; if the daemon is unreachable, the healthz gate fails and the runtime stays on noop.

• Cross-version compat. An M5 plugin against an M4 daemon sees sessions_schema_version: 1 but no tier_events_schema_version in healthz, so setRegisteredTelemetry is never called and the runtime stays on noop. An M5 daemon against M4 clients keeps shipping the same /v1/sessions/* surface untouched.

• Explicitly deferred to later: aggregation endpoints (/v1/tier-events/stats?...), real-time push (WS/SSE), server-side retention policy, client-side batching/coalescing, FK on session_id. M5 ships raw events + filters; everything on top of that is composable later.

M6 — Tool registry + invocation (shipped 2026-05-23)

• Daemon side: /v1/tools.

  • GET /v1/tools lists every registered tool sorted by name: { tools: [{ name, description, params_schema, handler, created_at, updated_at }] }. params_schema is a string-serialised JSON Schema; handler is a stable opaque dispatch ID (debug-only — invocation always goes by name).
  • GET /v1/tools/{name} fetches one (404 tool_not_found).
  • POST /v1/tools/{name}/invoke runs it. Body { arguments: {...}, session_id? } → { result: {...} } (200) or { error: { code, message } } (400 invalid_arguments / 404 tool_not_found / 500 handler_failed / 413 payload_too_large at the same 8 MiB cap as /v1/sessions/* and /v1/tier-events).
  • One tools table: (name PK, description, params_schema, handler, created_at, updated_at). Process-scoped, no session FK. Dispatch is a daemon-internal map[string]ToolHandler in toolhandlers.go; no external registration in M6 (no POST/DELETE on /v1/tools itself) — the catalog is seeded into the binary at boot.
  • Per-feature meta.tools_schema_version = 1, independent of the cross-language meta.schema_version = 1 (still M2-compatible with the TS @opencoti/memory reader).

• Two seed tools are UpsertTool'd at startup (idempotent — created_at fixed on first insert, updated_at bumps on re-seed):

  • opencoti_episodic_search — lexical LIKE search across the M4 session_messages table joined with sessions, newest-first. Params { query, limit?, session_id_excludes? }. The exclude list lets the plugin pass the live session ID so the model recalls other sessions, not echoes of the current one. The integrated analog of claude-hooks's episodic_server — same SQLite file the sessions API writes, no second daemon, no shell-out. FTS5/vector ranking is an M7+ optimisation.
  • opencoti_recall — vector recall over the M2 memory store, delegating to the same Store.Recall path /v1/memory/search uses. Params { embedding, collection?, k?, session_id? }. Takes a pre-computed embedding, not a query string — the daemon bundles no embedder, so the caller (the plugin, or an operator via curl) embeds the query and POSTs the float vector; length must equal the daemon's embedding_dim. session_id drives the M2 per-session ACL check. Coexists intentionally with @opencoti/memory's in-process __memory_recall: that one survives a daemon being down; opencoti_recall gives the same surface to plugins without direct DB access.

• TS plumbing. @opencoti/opencoti-server-client gains listTools + getTool + invokeTool; healthz() surfaces toolsSchemaVersion?: number as the plugin's feature gate. ServerTool is camelCase (paramsSchema ← params_schema), paramsSchema left as unknown (interpreted in the plugin).

• Design pivot continues: plugin path > surgical hook. A new module-level registered-server-tools.ts slot in @opencoti/tiers (mirrors M5's registered-telemetry.ts) holds the daemon tools as a Record<string, Tool>; runtime.ts merges it last into the tier tool list, next to synthetic-tier and memory-bridge tools:

  const tools = mergeTools(hook.prepared.tools, {
    ...syntheticTools(hook.input.sessionID, telemetry),
    ...memoryTools,
    ...(getRegisteredServerTools() ?? {}),
  })
  

  Server tools merge last so user-explicit + memory tools win on a name collision (none expected — server tools are opencoti_*, memory __memory_*, synthetic-tier __tier_*). Zero new opencoti-hook source markers — count unchanged from F3 M4/M5 (9 source + 4 JSON deps + 3 prose).

• The @opencoti/server-tools-plugin probes /v1/healthz (gated on toolsSchemaVersion >= 1), fetches /v1/tools once on boot, and builds an AI SDK dynamicTool per entry: params_schema round-trips through ai's jsonSchema() (no JSON-Schema-to-Zod reimplementation); each execute forwards to /v1/tools/{name}/invoke. Tool names are prefixed (opencoti_, idempotently). The session ID arrives via experimental_context on the single openStream streamText call and is forwarded as session_id. Invocations are awaited (not fire-and-forget — tool calls are model-blocking), but errors crash the tool call, not the session: 404→tool_not_found, 400→invalid_arguments, 5xx/network→tool_handler_failed, deadline→timeout (plugin-owned Promise.race, separate invoke_timeout_ms). Args over 1 MiB are pre-rejected client-side.

• Auto-wire via defaults. @opencoti/server-tools-plugin is in OPENCOTI_DEFAULT_PLUGINS. No reachable daemon → healthz gate fails → setRegisteredServerTools never called → runtime tool set unchanged.

• Cross-version compat. An M6 plugin against an M5 daemon sees no tools_schema_version in healthz, so the gate fails and the tool set is unchanged — a model query expecting the tool gets "I don't have that tool", no crash. An M6 daemon serves M5 clients the /v1/sessions/* and /v1/tier-events surfaces untouched.

• Explicitly deferred: external tool registration (POST/DELETE on /v1/tools — needs a per-tool ACL/owner model); streaming tool results; tool invocations as tier_events audit rows; an MCP wrapper for the registry; FTS5/vector ranking in episodic_search; an in-daemon embedder so opencoti_recall can take a query string directly.

M7 — pgvector backend parity (shipped 2026-05-23)

opencoti-server gains its second storage backend: PostgreSQL + the pgvector extension, via github.com/jackc/pgx/v5 (pure Go, no new cgo). *pgvector.Store implements the full interface family — Store + SessionStore + TierEventStore + ToolStore + SchemaInspector — so the HTTP layer, /v1/healthz, and the TS client are untouched. sqlite-vec stays the zero-dependency default; pgvector is for hosts that already run Postgres.

Schema-phrasing correction. The original M7 stub said "/v1/memory works against pgvector with the same schema as claude-hooks." That predated the M4–M6 buildout. opencoti's binding contract is now its own Store interface (collections + per-session ACL + sessions + tier_events + tools), which is a different data model from claude-hooks's flat memories + kg_* schema. So M7 implements opencoti's own model on a dedicated opencoti Postgres database, isolated from claude-hooks. "Same as claude-hooks" is reread as same storage technology (Postgres + pgvector), not the same tables.

• internal/store/pgvector/ mirrors internal/store/sqlitevec/ table-for-table in PG dialect, with three dialect differences:
  • the embedding lives inline on memories as a vector(dim) column (no separate vec0 virtual table); an HNSW vector_l2_ops index is created when dim ≤ 2000 (pgvector's HNSW ceiling) — correctness-neutral, perf-positive.
  • Recall pushes the ACL filter into the query (WHERE collection_name = ANY($readable) ORDER BY embedding <-> $q LIMIT k) — Postgres can filter + rank in one statement where sqlite-vec must overscan + join-back + post-filter. The <-> operator is L2, matching sqlite-vec's vec0 default, so MemoryHit.Distance stays comparable across backends.
  • Store dedups via INSERT … ON CONFLICT (collection_name, content_hash) DO NOTHING RETURNING id in one round trip.
• Per-feature schema versions (schema_version, sessions_/tier_events_/tools_schema_version, all 1) live in a meta key/value table, validated on open exactly as the sqlite-vec backend does — but they are pgvector's own contract (no cross-language reader, since pgvector is daemon-only).
• Shared conformance harness internal/store/storetest/ is the parity guarantee: a single RunConformance body (Memory, Sessions, TierEvents, Tools sub-suites) that both backends opt into via a thin conformance_test.go. Either backend drifting from the contract fails the same assertions. sqlite-vec runs it with t.TempDir; pgvector with a testcontainers fixture.
• Pure-logic lifted to internal/store/common.go (package store): the ACL resolver (ResolveAccessMode/CanRead/CanWrite), the collection/session validators, Sha256Hex, and the dim bounds — one cgo-free source of truth both backends share. sqlite-vec keeps its exported names as thin delegating wrappers, so the HTTP layer's sqlitevec.ValidateCollectionName/ValidateSessionID call sites are byte-identical.
• Backend selection is a serve-time flag, not a build flag: --backend sqlite|pgvector (default sqlite) + --pg-dsn (falling back to $OPENCOTI_PG_DSN). --db-path stays sqlite-only; --embedding-dim applies to both (locks the vector(dim) column). The interface var is assigned only on a successful open (avoids the typed-nil trap), so a pgvector open failure degrades to store_ok=false + a clear store_error rather than crashing. The Windows install command bakes --backend/--pg-dsn into the service args alongside the existing flags.
• healthz reports a redacted store_path for pgvector (pg://user@host:port/db, never the password).
• Tests: pgtest spins an ephemeral pgvector/pgvector:pg17 container via testcontainers-go and skips cleanly when Docker is absent (testcontainers.SkipIfProviderIsNotHealthy), so make test stays green on a Docker-less host. Both backends pass the full shared conformance suite; an end-to-end smoke (daemon → pgvector container) confirms collection create, store, and <-> - ranked recall over HTTP.
• Toolchain: the modern pgx / pgvector-go / testcontainers-go releases require Go 1.25, so the module's go directive and the host toolchain moved to go1.25 (latest stable). No build-posture regression — pgx is pure Go; sqlite-vec's existing cgo requirement is unchanged.
• Zero new surgical hooks. pgvector is additive Go inside opencoti/server/; no TS changes. The surgical-hook grep count stays at 18 (see docs/protocols/UPSTREAM_SYNC.md).

Deferred from M7: data migration between backends (a migrate subcommand — "Migration tools later"); a TS pgvector backend for the in-process @opencoti/memory (pgvector is daemon-only); external tool registration over HTTP (still security-gated, from M6); pgxpool tuning / read replicas.

M8 — Multi-session coordination (shipped 2026-05-23)

Goal G3: opencoti instances/sessions on one host see each other and cooperate. M8 ships the live-coordination half as an in-memory, ephemeral hub — deliberately store-independent (neither the Store interface nor either backend is touched), because presence and locks are runtime state that should not survive a daemon restart.

• internal/coord.Hub — backend-agnostic, one sync.RWMutex:

  • Presence: RegisterPeer / Heartbeat / DeregisterPeer / ListPeers, with a background TTL sweep that reaps peers whose heartbeat lapsed (default 45s) and emits peer.left. ListPeers also filters expired peers lazily.
  • Broadcasts: a pub/sub bus — Publish assigns a monotonic seq, appends to a bounded replay ring, and fans out non-blockingly to subscribers (a full subscriber channel is dropped + closed so the client reconnects with its last seq). Subscribe(since) atomically snapshots the replay backlog and registers for live events.
  • Advisory locks: try-only AcquireLock (reentrant-by-holder refresh; expired locks reclaimable) / ReleaseLock (holder-checked) / ListLocks. Lock transitions emit lock.acquired / lock.released.

• HTTP (internal/server/coord.go), hub injected via Options.Hub, 503 coord_unavailable when absent:

  Method + path	Purpose
  POST /v1/coord/peers	register/upsert presence → PeerInfo
  POST /v1/coord/peers/{id}/heartbeat	refresh TTL
  DELETE /v1/coord/peers/{id}	deregister
  GET /v1/coord/peers	list live peers
  POST /v1/coord/broadcast	publish {peer_id,topic,payload} → {seq}
  GET /v1/coord/events?since=&peer_id=	SSE event stream (the daemon's first)
  POST /v1/coord/locks/{name}	acquire {holder,ttl_ms?}; 200 or 409 lock_held
  DELETE /v1/coord/locks/{name}	release {holder}; 200 / 404 lock_not_held / 409 lock_not_holder
  GET /v1/coord/locks	list held locks

  SSE is viable because the http.Server sets no WriteTimeout; the handler exits on request-context cancellation so graceful shutdown releases it within the grace window. /v1/healthz gains coord_ok + coord_peers.

• TS client: registerPeer / heartbeatPeer / deregisterPeer / listPeers / broadcast / acquireLock / releaseLock / listLocks, plus subscribeCoordEvents — the client's first streaming method (reads response.body, parses data: frames).

• @opencoti/coordination-plugin: registers each session as a peer, heartbeats while active, deregisters on delete, subscribes to the event stream (SSE) to keep a live peer view, and advertises "N other active opencoti session(s)" in the system prompt. Auto-wired via the @opencoti/tiers default-plugins list — no surgical hook (grep count stays 18).

Deferred to F3 M9: opt-in shared context (a thin convention atop existing M2 global collections + per-session ACL — a session opts to expose a collection to peers); the sqlite↔pgvector migrate subcommand (open from M7); blocking/queued lock acquire (M8 is try-only). Cross-host federation remains an explicit F3 non-goal.

M9 — Opt-in shared context (shipped 2026-05-23)

Goal G3, the persisted-context half: a session exposes one of its session-scoped memory collections to peer sessions on the same host. The storage model from M2 already supports the grant, so M9 adds no new Store method, no new schema, no new surgical hook (grep count stays 18). It is a thin convention bridging two things that already exist — the persisted per-session ACL (SetSessionACL) and M8's ephemeral coord bus.

• internal/share.Manager — in-memory registry + one goroutine:

  • Share(collection, owner, mode) records the share, grants read-ACL to every live peer != owner via SetSessionACL, and publishes collection.shared on the hub.
  • It subscribes to the hub and, on peer.joined, grants every active share to the newcomer — so peers that join after a share still get access (auto-grant via the bus, dogfooding M8's SSE).
  • Unshare(collection, owner) revokes live peers (writes mode none, the non-owner default) and publishes collection.unshared.
  • Persisted vs ephemeral: the ACL grants persist (they survive a restart); the "keep auto-granting new joiners" intent is in-memory and lost on restart by design — existing grants remain, but the owner must re-share to resume auto-granting. This keeps M9 storage-free.

• Recall "just works": once a peer holds an r grant, its no-filter Recall includes the shared collection automatically (M2's readable-collections resolution), so no recall-path change is needed.

• HTTP (internal/server/share.go), Manager constructed in server.New() when Store+Hub are present, released on Shutdown; 503 share_unavailable when the hub is absent:

  Method + path	Purpose
  POST /v1/memory/collections/{name}/share	owner-only share {owner_session, mode?} (mode default r) → {granted}
  DELETE /v1/memory/collections/{name}/share	withdraw {owner_session} → 200
  GET /v1/memory/shares	list active shares

  Owner-only: the handler looks the collection up via ListCollections (no filter) and requires scope == session and session_id == owner_session; global collections (already r-for-all) are rejected not_shareable, a different owner not_owner (403).

• TS client: shareCollection / unshareCollection / listShares

  • SharedCollectionInfo.

• @opencoti/coordination-plugin (extended in place — no new plugin): consumes collection.shared / collection.unshared to advertise peer-shared collections in the system prompt, and gains an opt-in share_session_collection flag (default false) that shares the session's own collection (sessionCollectionName(id)) on session.created and unshares it on session.deleted (best-effort; a not-yet-created collection's 404 is swallowed).

Deferred to F3 M10: the sqlite↔pgvector migrate subcommand — not a cutover: both backends are first-class and may run in parallel; migrate is an idempotent, re-runnable, either-direction copy (--from/--to, dedupe on content_hash, upsert PKs) that tops up a parallel target, never abandoning the source. Also deferred: revoke-on-peer-leave cleanup (M9 keeps grants on peer.left, since peers may return) and wildcard / group ACLs.

M10 — sqlite↔pgvector migrate (shipped 2026-05-24)

A daemon-internal migrate subcommand that copies data between the two first-class backends. It is not a cutover: both backends stay first-class and may run in parallel (two daemons, or alternating --backend). migrate is an idempotent, re-runnable, either-direction, selectable copy that tops up a target (dedupe on natural keys), never "move then abandon source". Additive Go only — no plugin, no HTTP surface, no TS client, no surgical hook (grep count stays 18).

Use cases (maximum flexibility):

1. Scale-up (primary). Start on sqlite; migrate everything (collections, memories+embeddings, ACL, sessions, session_events, session_messages, tier_events, tools) into pgvector; then switch the primary backend by changing the serve flag to --backend pgvector. "Switch primary backend" is operational, no extra code: run the full migrate, then change --backend.
2. Way back. Same command with --from/--to swapped.
3. Single memory container / additional partial backend. Copy only specific collection(s) with --collection, so e.g. pgvector holds just certain memories while sqlite keeps the rest. The two backends coexist, each holding different data.
4. Additive top-up. Re-running, or copying into a populated target, merges idempotently (natural-key dedupe).

Selection model — two orthogonal selectors:

• --include <csv> of sections: memory,sessions,tier-events,tools. Default (unset) = all four (full dataset). The memory section carries collections + their memories + their ACL rows.
• --collection <name> (repeatable) restricts the memory section to those collections only. When --collection is given and --include is unset, the default narrows to memory-only (the single-container case).

So: full switch = no selectors; single container = --collection notes; logs-only = --include sessions,tier-events.

Copy phases (FK-respecting; each gated by the selection):

Section	Phase	Export	Import
memory	collections	ListCollections (filtered)	PutCollection
memory	memories	ListMemories (per coll, keyset-paged)	PutMemory
memory	session_acl	ListSessionACLs	SetSessionACL
sessions	sessions	ListSessions(+deleted)	UpsertSession
sessions	session_events	ListSessionEvents/sess	PutSessionEvent
sessions	session_messages	ListSessionMessages/sess	UpsertSessionMessages
tier-events	tier_events	ListTierEvents (batched)	PutTierEvent
tools	tools	ListTools	UpsertTool

Both backends are the same concrete *Store implementing every feature interface, so migrate opens each side as storepkg.Store, type-asserts the optional MigrationStore, and copies a section only if both sides implement it. Per-phase counts (scanned / inserted / skipped) print at the end; --dry-run reads sources and reports would-copy counts, no writes.

Idempotency / dedup. Memories dedupe on (collection_name, content_hash). The append-only audit logs (session_events, tier_events) have no natural key (autoinc id only), so their idempotent import dedups on an insert-if-no-identical-row check (all business columns match) — re-running produces no duplicates. A second full run reports 0 inserts everywhere.

Two backend-specific wrinkles (both resolved):

• sqlite-vec embeddings are an opaque BLOB. vec0 memory_vecs stores the embedding as a blob and the Go binding ships SerializeFloat32 but no deserialize. The on-disk format is plain little-endian float32 (binary.Write(buf, LittleEndian, vector)), so deserializeFloat32 reverses it with binary.Read at the locked dim — byte-faithful, no re-embedding. pgvector exports via pgvector.Vector.Scan + .Slice().
• The public Store() write path enforces ACL (a global collection is r-for-all → ErrWriteDenied), so migrate cannot reuse it for imports. The new MigrationStore.PutMemory is ACL-free and timestamp- preserving.

Fidelity contract (documented caveat). Preserved exactly: collection created_at, memory ts+content+embedding (byte-identical), session created_at/deleted_at, message created_at/finished_at, event occurred_at/ts, all ACL modes. Rewritten to migrate-time (reused upserts stamp now()): sessions.updated_at, tools.created_at/ updated_at — acceptable "last-written" fields (tools are also re-seeded at startup). A byte-identical PutSession/PutTool is deferred.

The new MigrationStore optional interface (in store.go, mirroring the SchemaInspector/SessionStore convention) is exercised by the storetest conformance harness on both backends — the round-trip asserts the exported embedding is byte-identical, validating the sqlite LE deserialize and the pgvector scan together. Verified end-to-end across a sqlite→pgvector→sqlite hop: recall on the round-tripped file returns distance 0 for the exact source vector.

Deferred to F3 M11+: continuous-sync / daemon mode (migrate is one-shot CLI only), cross-host federation (explicit F3 non-goal), per-session selective log copy (selective granularity is per-collection for memory + section toggles for the rest), and the byte-identical updated_at/tool-timestamp imports noted above.

Open questions

• Do we ship the server inside the opencoti binary or alongside it? Alongside (separate binary) is cleaner; explore a single-fat-binary option as an opt-in.
• Authentication. Local UDS owner is the user; over loopback TCP we need a shared secret. Default to UDS.
• claude-hooks coexistence. If a user runs both Claude Code (with claude-hooks) and opencoti (with opencoti-server) on the same host, the two memories should not collide. They use different stores and different schemas by default; a separate migration tool can bridge them if the user wants.

Risks

• Two persistence backends doubles the test matrix. Mitigation: a storage trait + a shared conformance test.
• Drift from claude-hooks features. Mitigation: don't try to replicate feature-for-feature; replicate goals. claude-hooks remains the reference for Claude Code; opencoti-server can diverge where it makes opencoti better.