docs/features/memory_embedder.md

F4 — Memory / Embedder

Parent: ../MASTER_PLAN.md Status: planning (M0+M1+M2 done — 2026-05-22) Owner: TBD Reference: /shared/dev/claude-hooks (the embedder lifecycle is mirrored from claude-hooks' Python implementation)

Problem

opencoti needs persistent memory on day-one — well before F3 (opencoti-server, the Go companion daemon) is ready. Memory needs an embedder, and the embedder needs to work without requiring the user to install a separate runtime. The same constraint that drove F2 (bundle llamafile so opencoti runs out of the box) drives F4: ship a prebuilt embedder llamafile, manage it with a small TypeScript launcher, default to CPU, allow opt-in GPU.

There's a second, sharper problem the user surfaced from real-world claude-hooks operation: the canonical embedder (qwen3-embedding-0.6b-16k.llamafile, ~1.5 GB on disk for a 700 MB model) allocates 3.32 GB RSS at idle. The bottleneck is per-slot context buffer allocation, not the autoregressive KV cache. The embedder server runs --parallel 2 --ctx-size 16384 and pre-allocates the full 16K context for both slots up front. Live workload shows most requests at 7–56 tokens with only 1 slot active most of the time — so the eager allocation pays full price for capacity that is almost never used.

F4 ships the embedder, and ships a lazy-slot-context patch against the vendored llamafile/llama.cpp tree so the embedder's RSS scales with actual demand instead of slot_count × n_ctx_slot.

Goals

• G1. TypeScript package @opencoti/embedder that owns the embedder lifecycle: spawn, health-probe, idle-reap, GPU/CPU mode, embed(text) / embedBatch(texts).
• G2. First-run setup downloads the prebuilt asset (qwen3-embedding-0.6b-16k.llamafile, dim 1024, default port 47092 in opencoti's reserved 47000-48000 range) into the opencoti data dir. Resume + SHA256 verify.
• G2a. Daemon-launch policy (project-wide convention). Any opencoti or opencoti-server daemon that opens a listening port MUST: (1) pick its default port from the 47000-48000 range — opencoti must never collide with claude-hooks' 38000-39000 / 18790-18811 ranges so the two systems coexist on one host; (2) ask the user at first-run setup where to bind — 127.0.0.1 (default), all interfaces, or a specific IP — and (3) propose the default port but allow the user to override it before persisting the choice. The setup flow validates that the chosen port is free and within range before writing the config.
• G3. GPU mode = auto-detect with CPU fallback; default = CPU. Reuses F1 M5.7's gpu-detect chain (nvidia → amd → vulkan).
• G4. Composite fallback chain: an EmbedderClient interface + a CompositeEmbedder that tries primary then fallback on EmbedderError, with a dim-match assertion to prevent vector-space corruption.
• G5. Lazy-slot-context patch (0001-lazy-slot-context.patch) against the vendored llamafile so RSS scales with real demand: defer per-slot allocation, grow on demand, shrink on idle.

Non-goals

• Replacing claude-hooks for users who already run it on the same host. opencoti's embedder is independent and uses its own port + data dir.
• Bundling weights. The model GGUF is inside the .llamafile asset; we don't ship a separate GGUF download.
• A full vector store + recall pipeline. F4 M5 wires recall + store as a thin seam; the real vector-store ownership lives in F3 (opencoti-server) when it ships.

Default asset

• Repo: mann1x/claude-hooks releases — transitional pin, two-stage migration:
  • Stage 1 (immediate): mirror the byte-identical asset to a mann1x/opencoti release (proposed tag: embedder-v1.4.0) and flip just the repo field in vendors/pin/embedder.txt. Same SHA, no rebuild — decouples opencoti's release stream from claude-hooks'.
  • Stage 2 (after F4 M3 lazy-context patch ships): rebuild the embedder composite-llamafile to include the lazy- slot-context patch. The new binary has a different SHA; tag + sha256 in the pin file both bump (proposed tag: embedder-v1.5.0). F4 M6 (release bundling) owns the apply-patches → make → zipalign-with-GGUF-and-args pipeline. Without Stage 2, the embedder still benefits from the lazy-context patch only when run against a user-supplied locally-built embedder binary — not the prebuilt asset.
• Tag: v1.4.0
• Asset: qwen3-embedding-0.6b-16k.llamafile
• SHA256: 414f616689aaba44f6982b474918e8549ad764c03b2f82f6b56a5d6e582ef59b
• Pinned in: vendors/pin/embedder.txt
• Default port: 47092 (opencoti's reserved 47000-48000 range — see G2a).
• Endpoint: POST /embedding (llama.cpp server style, not OpenAI-compat). Request: {"content": "<text>"} or batch {"content": ["t1", "t2"]}. Response: {"embedding": [...]} (single) / [{"embedding": [...]}, ...] (batch).

Design sketch

Layout

packages/opencoti-embedder/           # NEW
├── package.json
├── tsconfig.json
├── src/
│   ├── index.ts
│   ├── client.ts                     # EmbedderClient interface + types
│   ├── llamafile-embedder.ts         # POST /embedding HTTP client
│   ├── composite.ts                  # primary → fallback chain
│   ├── manager.ts                    # process spawn / health / idle reap
│   ├── gpu-mode.ts                   # CPU | auto resolution
│   ├── download.ts                   # asset fetch + SHA256 verify
│   └── config.ts                     # Zod schema for embedder config
├── script/
│   └── smoke.ts
└── test/

vendors/
├── pin/
│   └── embedder.txt                  # NEW (M0)
└── patches/
    └── llamafile/
        └── 0001-lazy-slot-context.patch   # NEW (M3)

Reused, not rebuilt

• @opencoti/tiers/registry/gpu-detect — nvidia/amd/vulkan probe chain (F1 M5.7).
• @opencoti/tiers/registry/hf-download — resume + SHA256 verify download. To be generalized into a shared @opencoti/tiers/util/download helper consumed by both HF downloads and GitHub Release downloads.

Milestones

M0 — Vendoring decision + asset URL pin (done — 2026-05-22)

• vendors/pin/embedder.txt records repo / tag / asset / SHA256.
• Decision recorded: pin claude-hooks v1.4.0 directly until opencoti has its own release stream. Switch the pin URL to mann1x/opencoti when that exists.
• This feature plan + F5 plan + MASTER_PLAN.md updated together in the same commit.

M1 — Embedder package, CPU mode, smoke green (done — 2026-05-22)

• @opencoti/embedder shipped with client interface, llamafile-embedder HTTP impl (POST /embedding, response-shape tolerant for A/B/C/D variants seen in the wild), download.ts (resume + SHA verify, builds the canonical github-release URL from the pin), manager.ts (spawn + /health poll + LRU idle reap @ 5 min default + idempotent stop), config.ts (Zod strict schema, defaults claude-hooks- parity), gpu-mode.ts (CPU default + auto-resolution), composite.ts (primary→fallback chain with dim-match guard).
• Shared downloadWithResume helper extracted to @opencoti/tiers/util/download; hf-download.ts refactored to consume it. New subpath exports: @opencoti/tiers/util/download, @opencoti/tiers/registry/gpu-detect, @opencoti/tiers/registry/hf-download.
• script/smoke.ts downloads the asset to <repo>/.opencoti/embedder/ (skips on SHA match), spawns CPU-mode, embeds "hello world", asserts dim=1024 + L2-norm. Live smoke run is opt-in (asset is 1.5 GB).
• 52 unit tests across 7 files. Hook footprint unchanged (still 4 opencoti-hook: markers).
• Out of scope (deferred to M5): opencode session injection, recall-into-prompt, vector store.

M2 — Baseline measurement (done — 2026-05-22)

Ships two scripts and three fixture files:

• script/generate-corpus.ts — deterministic 48-prompt corpus generator spanning 8-3000 approx_tokens. Buckets: 10 short (8-32 tok), 11 medium (33-128), 19 medium-long (129-512), 6 long (513-2048), 2 very-long (2049-8192). Original draft had 5 in the very-long bucket; 2 mega prompts (60+ LOREM repeats) aborted past the 60s embedder request-timeout during the live baseline run, so they were dropped from the generator.
• script/measure-baseline.ts — boots the embedder via the M1 manager (or attaches to a running instance with --use-running <url> --pid <int>), captures RSS from /proc/<pid>/status, extracts allocation lines from the embedder startup log, embeds the corpus, writes the fixture.

Fixtures (in test/fixtures/):

• quality-prompts.json — 48 prompts, deterministic.
• quality-baseline.json — 48 vectors (dim 1024) M3 must reproduce to cosine ≥ 0.999.
• baseline-meta.json — RSS + allocation breakdown + startup-log excerpt.

Live measurement (2026-05-22, attached to claude-hooks embedder, parallel=4 default, ctx=16384, CPU mode):

Component	Size
Model weights (CPU_Mapped)	603.87 MiB
KV cache (all slots)	1792.00 MiB
Compute buffer	330.24 MiB
Output buffer	2.33 MiB
RSS idle (process VmRSS)	2.52 GB
RSS peak (after 48-prompt run)	3.81 GB
Avg latency per prompt	5352 ms
Total corpus embed time	256.9 s

Startup-log key lines (captured in baseline-meta.json): llama_context: n_ctx = 16384, llama_kv_cache: CPU KV buffer size = 1792.00 MiB, sched_reserve: CPU compute buffer size = 330.24 MiB, server_main: embeddings enabled with n_batch (2048) > n_ubatch (512) → setting n_batch = n_ubatch = 512 to avoid assertion failure, slot load_model: id 0/1/2/3 | new slot, n_ctx = 16384. The binary defaults to parallel=4 with no --parallel override.

The earlier user observation of 3.32 GB RSS came from a 2-slot config explicitly setting --parallel 2 --ctx-size 16384. The M3 patch targets that production-shape config:

• Pre-patch per-slot KV allocation = n_ctx_slot × 2 (K+V) × layers × per-token-dim ≈ 448 MiB / slot in CPU mode, allocated eagerly for every slot at startup.
• M3 target: per-slot allocation deferred to first prompt, sized to --slot-initial-ctx 4096 (~112 MiB), grown on demand up to 16384, shrunk back to 4096 after --slot-shrink-idle-ms. Combined with 2 slots + model + compute, idle RSS target lands near 1.0 GB.

Two corpus prompts intentionally dropped: lorem-mega-00 and lorem-mega-01 (60+ LOREM repeats) aborted past the embedder's 60s default request-timeout on the CPU-mode production instance — n_ubatch is forced to 512 in embedding mode, so 9000+ token requests need ~18 forward passes that exceed 60s. Not an M3 concern (the patch addresses allocation, not throughput).

M3 — Lazy-slot-context patch

Architecture reference: ADR 0001 — Lazy slot-context allocation. The ADR documents the llama.cpp KV-cache layout, the three+1 orthogonal axes of "lazy," the four-phase rollout (defer / grow / shrink / per-stream split), the CLI surface, and the F5 milestone hand-offs that depend on the allocator hooks landed here. Implementation is multi-session per user directive B2.

Phase status (2026-05-23):

• Phase 1 — deferred zero-fill: shipped as vendors/patches/llamafile/0001-lazy-slot-context-defer.patch. Patch applies cleanly; build succeeds on x86_64 + aarch64; smoke validation on Qwen2-Math-1.5B confirms the KV cache zero-fill deferred until first batch startup log line and the matching ensure_cleared: ... (deferred) first-batch log line. Bench against M2 corpus + RSS table for the embedder workload pending (Phase 1 alone is a cold-start win; the headline warm-state RSS savings come in Phases 2-3).
• Phase 2 — grow on demand + partial zero-fill: shipped as vendors/patches/llamafile/0002-lazy-slot-context-grow.patch. Adds --slot-initial-ctx N (default 0 = Phase 1 behavior) that caps initial KV-cell allocation per sequence; find_slot honors the soft cap; prepare() grows on overflow up to kv_size_max via next-pow2 geometric ramp; ensure_cleared partial-memsets only the [n_cells_cleared, target) range via ggml_backend_tensor_memset. With --slot-initial-ctx 4096 for the production embedder, warm-state RSS scales with actual workload instead of the slot_count × n_ctx_slot product. Bench numbers vs M2 corpus pending.
• Phase 3 — shrink on idle: shipped as vendors/patches/llamafile/0005-lazy-slot-context-shrink.patch. Adds --slot-shrink-idle-ms N (default 30000); per-stream idle timer sweeps at init_batch and reclaims buffers back to --slot-initial-ctx after the timeout. Bench delta: 391.7 MB returned to kernel on the CPU embedder after 32 s idle (1.68 GB peak → 1.30 GB post-shrink, cosine 0.999797 vs M2 baseline). CPU-only — the shrink is gated on ggml_backend_buffer_is_host() inside opencoti_decommit_layer_range() because madvise(MADV_DONTNEED) is a no-op against device-backed pages. On GPU (-ngl > 0) the shrink_if_idle log marker still fires and the soft-cap bookkeeping rewinds, but the CUDA/ROCm/Vulkan backend buffer is not freed and VRAM stays at the peak the workload reached. The GPU-shrink path is tracked at F5 M2 (HeadInfer), not Phase 4.
• Phase 4 — per-stream tensor split: DEFERRED (task #109, target F5 M2). 2026-05-23 user decision after planning-session exploration showed Phase 4 is a deep refactor (per-stream tensors + per-stream backend buffers + reworked cpy_k/cpy_v scatter ops that currently rely on ggml_reshape_2d(k, n_embd, kv_size*n_stream) to flatten all streams into a single ggml_set_rows) and only adds value in non-unified mode. The embedder ships unified by default and already gets all Phase 1-3 wins with the contiguous tensor intact. Phase 4's real beneficiary is F5 (HeadInfer's per-head GPU/CPU split + PolyKV's shared pool). Full implementation map persisted in docs/decisions/0001-lazy-slot-context.md §Phase 4 — implementation map for the F5 owner to pick up cold. Not the GPU-shrink fix — that's F5 M2 (HeadInfer), independently of Phase 4.

• vendors/patches/llamafile/0001-lazy-slot-context.patch.
• Behavior:
  • Defer per-slot KV/compute-buffer allocation until slot's first real prompt.
  • Initial allocation rounded up to --slot-initial-ctx (default 4096).
  • Grow on demand (next power-of-two), cap at n_ctx_slot.
  • Shrink on idle (--slot-shrink-idle-ms, default 30000): free buffers and return to initial-ctx after the timeout, otherwise the first 16K request permanently sticks the slot at 16K and the lazy-allocation win evaporates.
• Bench (perf/llamafile/embedder-rss.bench.ts): RSS at idle / after 8-token request / after 16K request / after idle timeout + 8-token request. Quality: cosine ≥ 0.999 vs M2's baseline.
• Header: Milestone: F4 M3, Upstreaming: TBD — propose to llamafile + llama.cpp upstream once bench numbers are public.

M4 — Auto-GPU mode + composite fallback

• gpu-mode.ts adds "auto": probe nvidia → amd → vulkan; if VRAM ≥ 1.6 GB, launch with --gpu auto + n-gpu-layers fully offloaded; otherwise silent CPU fallback.
• CompositeEmbedder ships: primary (local llamafile) + fallback (cloud-route delegate, when Tier 1 is configured). Dim-mismatch assertion lives here.

M5 — Opencode integration: recall + storage seam (2026-05-23 — shipped, rescoped mid-milestone)

Original scope: one synthetic tools pair + one surgical session-end hook + setup-flow listen-address question. Actual scope after the user rescope (verbatim): "memory store and recall is going to be another crucial point, we support sqlite + sqlite_vec as base memory backend, managed by opencoti-server. opencoti-server will manage optional sharing on the network. the pgvector db is an add on memory connector. both works in parallel. memory in opencoti can be managed per-session. user can manage which memories to use and list, delete, connect to the memories via the harness menu. shared memory for all sessions, specific memories can be added and or connected. for each session the user can decide which one to read/write/rw"

Path taken (after user decision): in-process M5 now, refactor to opencoti-server later. The opencoti-server piece is queued for F3 once F4 closes. The data model and ACL contract land here so they survive that refactor unchanged.

What M5 actually shipped:

• M5-A — @opencoti/memory package. SqliteVecStore over bun:sqlite + sqlite-vec 0.1.9 vec0 virtual table. Embedding dim locked at DB creation (default 1024). Schema: collections, memories (with content_hash UNIQUE for idempotent insert), memory_acl (per-session override), vec_memories (vec0 virtual table with FLOAT[dim]). Default DB path ~/.opencoti/memory/state.db. ACL model: explicit overrides win; defaults are "rw" for global collections, "rw" for the owning session of a session-private collection, "none" otherwise. The resolver lives in pure code (resolveAccessMode, canRead, canWrite) and is exhaustively tested.
• M5-B — synthetic tools. __memory_recall(query, k?, collections?), __memory_store(content, collection?), __memory_list(). All three are ACL-aware (sessionID is the actor) and best-effort — embedder failure returns {ok: false, error: "embedder_unavailable"} rather than throwing. ensureSessionCollection is idempotent.
• M5-C — runtime wire-up via dynamic-import bridge. @opencoti/tiers/src/memory-bridge.ts resolves @opencoti/memory + @opencoti/embedder via dynamic import to break the tiers→memory→embedder→tiers cycle. Wired into runtime.ts:maybeRoute via mergeTools (no new surgical hook — extends an existing additive seam). The default embedder is createDefaultEmbedFn (loopback LlamafileEmbedder on 47092). Workspace dep @opencoti/memory added on packages/opencode to make the dynamic import resolvable; registered in UPSTREAM_SYNC.md as memory-tools-resolvability (dep).
• M5-D2 — @opencoti/memory-plugin (opencode SERVER plugin). Two hooks: experimental.chat.system.transform appends a system-prompt suffix advertising the memory tool surface + listing the session's accessible collections; event listens for session.idle and, when auto_ingest: true (default FALSE), ensures the session-private collection. Defaults reflect the rescoped vision: advertise=true (the point of loading the plugin), auto_ingest=false (the agent already has __memory_store).
• M5-D1 — @opencoti/tui-memory (opencode TUI plugin). One command opencoti.memory opens a DialogSelect listing every collection (global + session-private), with per-session resolved-mode annotations when in a session route. Drill-down per collection offers: ACL toggle r/w/rw/none (session-only), DialogConfirm-guarded delete, Create flow (DialogPrompt name → scope DialogSelect). Lazy store open, closed cleanly on api.lifecycle.onDispose.

Default-plugin auto-wiring (2026-05-23 follow-up to user directive: "opencoti will need to wire them by default"): The opencoti plugins remain opt-in for upstream-opencode users (they stay out of upstream's plugin list unless explicitly declared), but opencoti's distribution auto-wires them whenever a project has any opencoti.* config section. Implementation in @opencoti/tiers/default-plugins: a pure helper applyDefaultPlugins(cfg) gap-fills @opencoti/memory-plugin, @opencoti/tui-memory, @opencoti/tui-tiers into the final plugin_origins list — user-declared entries (including their options) are preserved untouched. Wired via two surgical hooks in packages/opencode/src/config/config.ts (import + call-site, registered as opencoti-default-plugins). Net change vs M5 substance: TWO new opencoti-hook markers (still tagged + in registry). With opencoti disabled (no cfg.opencoti block) the helper is a no-op and the byte-output is identical to upstream.

What M5 deliberately did NOT ship:

• The session-end surgical hook (the original plan). Replaced by the optional event: "session.idle" listener in @opencoti/memory-plugin with auto_ingest: false default. Net new opencoti-hook: markers from M5 substance + follow-up: TWO (the opencoti-default-plugins import + call-site). The memory-tools-resolvability dep is a registry-only entry (no source marker — JSON has no comments).
• The pgvector add-on connector. Deferred — the in-process backend is sqlite-vec only for now. Pgvector becomes a parallel MemoryStore implementation once opencoti-server (F3) lands.
• The network-sharing layer. That's opencoti-server's job (F3).
• The "Clear ACL override" UI option in the TUI panel. The MemoryStore interface does NOT currently expose a clear-ACL method — adding it would require a new schema/interface revision (deferred to a future minor bump).
• M5-E — setup-flow embedder daemon listen address question (shipped 2026-05-23). @opencoti/tiers/cli/setup-flow extended with an embedder listen-address prompt in finalizePatch. Steps: (1) DialogSelect over loopback / all-interfaces / specific-IP / skip; (2) if specific-IP, prompt for the IP with an IPv4 validator; (3) text prompt for the port, validated against the opencoti 47000-48000 range; (4) port-free check via injected isPortAvailable(host, port) (live: TCP bind probe in setup.ts) — busy port logs a warn and asks the user whether to proceed anyway; "no" recursively re-asks the bind block. The chosen {host, port} lands in ~/.config/opencode/opencode.jsonc under opencoti.embedder.host + .port via the jsonc-parser modify-then-applyEdits path (comments preserved). A new embedderConfigured sentinel (true when BOTH host AND port already present in the user's config) gates the prompt on re-runs. 16 new tests across the flow + the validatePort and isValidIPv4 pure helpers.

M6 — Production llamafile build pipeline (2026-05-23 — promoted ahead of M5)

Driver: user directive 2026-05-23 — the post-F4 M4 state (prebuilt claude-hooks asset + manual zipalign of ggml-cuda.so + cross-repo pin dependency) is "patchwork […] not acceptable for production". M6 was originally scoped to the embedder composite alone; the expanded M6 below covers all three backends (CUDA / ROCm / Vulkan) and the local-asset modes the embedder downloader needs to consume an opencoti-built artifact.

M6 lands before M5 (opencode session-end hook) because there's no point storing memories on the back of a launcher binary that doesn't engage the M3 lazy-context patch.

What M6 ships:

1. Multi-backend DSO builders in packages/opencoti-llamafile/script/build-pipeline.ts: runCuda / runRocm / runVulkan / runAllBackends, driven by a shared buildBackend(spec) helper.

   • Each backend has a toolchain probe: OPENCOTI_CUDA_PATH/bin/nvcc (default /usr/local/cuda-12.6 — CUDA 13.x is known-broken with cuda.sh), OPENCOTI_ROCM_PATH/bin/hipcc (default /opt/rocm), OPENCOTI_VULKAN_SDK env / system glslc.
   • Soft-fail per backend: a host without ROCm produces a clean CUDA-only build instead of failing the pipeline.
   • Each built DSO is staged at vendors/dist/llamafile/<backend>/<ver>/ggml-<backend>.so AND mirrored to ~/.llamafile/v/<ver>/ so devs running the thin binary directly still get GPU support locally.

2. package subcommand in the same script: Zipaligns the thin patched binary with every staged DSO via the vendored vendors/sources/llamafile/o/third_party/zipalign/ zipalign -j0 invocation. Always emits dist/llamafile/opencoti-llamafile-<ver>-<arch>.llamafile plus a sibling MANIFEST.json (artifact SHA + per-DSO SHA + patches list).

   • With --with-model PATH ALSO emits dist/llamafile/opencoti-embedder-<ver>-<arch>.llamafile by copying the bare and zipalign-embedding the GGUF + a .args file generated from the F2 M3 canonical embedder args contract (-m /zip/<basename> --server --embedding --pooling last --ctx-size 16384 --parallel 2 --slot-initial-ctx 4096 --slot-shrink-idle-ms 30000).
   • --with-args PATH overrides the generated .args.
   • --cpu-only produces a binary with NO DSO embedded (for CI smoke on hosts without GPU).

3. Pure helpers in packages/opencoti-llamafile/src/build-pipeline.ts (kept side-effect-free per F2 M3 convention): Backend type + ALL_BACKENDS, dsoFilename, stagingDir, artifactDir, readBackendArtifacts, sha256OfFile, composeEmbedderArgs, parseBuildArgv, ArtifactManifest type. 19 new unit tests under test/build-pipeline.test.ts.

4. @opencoti/embedder downloader local-asset modes (src/download.ts):

   • file: prefix on the pin's repo field — bytes come from disk; SHA still verified.
   • OPENCOTI_EMBEDDER_LOCAL=<abs-path> env override — bypasses pin path entirely; SHA still verified.
   • Both routed through a shared localCopyPath helper; the HTTPS+resume path is unchanged. 12 new tests in test/download.test.ts.

5. Hook footprint unchanged. M6 is pure build orchestration + downloader extension; no surgical hooks in upstream opencode.

Deferred to F4 M7 (release bundling):

• GitHub Releases publish pipeline. Producing artifacts under dist/llamafile/ is M6; uploading them as mann1x/opencoti release assets and flipping vendors/pin/embedder.txt to mann1x/opencoti (Stage 2 of the migration the existing pin file header describes) is M7. The pin migration is a one-line change once the release exists.
• Metal / macOS backend. vendors/sources/llamafile/llamafile/ metal.c is macOS-only; M6 ships Linux-only since the build host is solidPC. Needs a Darwin CI runner.
• aarch64 / multi-arch DSO orchestration. cosmocc APE is dual- arch by design, but ggml-*.so are arch-specific. The <arch> token in the output filename is anticipatory; M6 ships x86_64 only.

M7 — Release bundling + dev-cut workflow alignment with claude-hooks

The build pipeline (M6) produces dist/llamafile/*.llamafile artifacts. M7 wraps that with the commit-the-contract, gitignore-the-bytes release scheme claude-hooks uses, plus a release-cut subcommand that prepares (but does NOT auto-publish) a mann1x/opencoti GitHub Release.

What's tracked (committed):

• vendors/llamafile/README.md — audit-trail doc describing the pinned upstream + the contract layout. Mirrors claude-hooks's vendor/llamafile/README.md.
• vendors/llamafile/LICENSE.upstream — single-file copy of the four upstream LICENSE files (Mozilla-Ocho/llamafile Apache-2.0 + llama.cpp/whisper.cpp/stable-diffusion.cpp MIT × 3) at the pinned commit. Survives independently of the submodule.
• vendors/llamafile/SHA256SUMS.composite — the contract. sha256sum-compatible lines listing every released composite artifact. Refreshed by build:llamafile:release-cut. Verified by the embedder downloader at install time.

What's gitignored (reproducible from source):

• vendors/dist/llamafile/<backend>/<ver>/ggml-*.so — per-backend staging DSOs.
• dist/llamafile/*.llamafile — final composite artifacts.
• dist/llamafile/*.MANIFEST.json — per-artifact manifests with embedded-backend metadata.

Release-cut workflow (matches claude-hooks's make -C vendor/llamafile/dist):

1. bun run build:llamafile — patched binary.
2. bun run build:llamafile:all-backends — every available DSO.
3. bun run build:llamafile:package -- --with-model PATH — bare
   • composite artifacts.
4. bun run build:llamafile:release-cut — refreshes the SHA contract; prints the suggested gh release create invocation (the actual publish is a user-confirmation step, not automated).
5. User executes gh release create + gh release upload, then commits the refreshed SHA256SUMS.composite + the bumped vendors/pin/embedder.txt.

Downloader behavior: when shaContractText is provided to downloadEmbedder, the pin's SHA is cross-checked against the contract before any download or local-copy. Disagreement is a hard error with a "redownload or rebuild" breadcrumb (matches claude-hooks's install.py). An asset not present in the contract is a soft warning, not an error — supports the transitional state where the pin still references claude-hooks pre-flip.

What's NOT in M7:

• Actual first mann1x/opencoti GitHub Release publish — that's the user-confirmed external step the scaffolding enables.
• Pin flip from mann1x/claude-hooks → mann1x/opencoti — only meaningful after the first release exists.
• macOS / aarch64 — same constraints as M6.

Critical files (M6):

• packages/opencoti-llamafile/script/build-pipeline.ts — new subcommands.
• packages/opencoti-llamafile/src/build-pipeline.ts — pure helpers + types.
• packages/opencoti-llamafile/test/build-pipeline.test.ts — helper tests (19 new, 46 total).
• packages/opencoti-embedder/src/download.ts — local-asset modes.
• packages/opencoti-embedder/test/download.test.ts — local- asset tests (12 new, 13 total for download).
• docs/features/llamafile_build.md — toolchain + pipeline documentation.
• Root package.json — new build:llamafile:{rocm,vulkan, all-backends,package} scripts.
• .gitignore — vendors/dist/, /dist/llamafile/.

Surgical-hook footprint

• M1–M4, M6: zero new hooks.
• M5: exactly one new hook (session-end memory ingest).

Open questions

• Release stream ownership. Decision (M0): pin claude-hooks v1.4.0. When opencoti's own release cadence starts, switch the pin URL to mann1x/opencoti. Same SHA + same asset on disk; just a different repo field.
• Composite-llamafile build. F4 consumes the prebuilt asset. If F4 M3's lazy-context patch lands and we want it in the embedder binary too, we need to build our own composite-llamafile (vendored llamafile + Qwen3 GGUF + .args). That's potentially a small new milestone (F4 M6) that overlaps with F2 M6 (release artifact bundling).
• Idle-reap vs always-on for the manager. 5-min idle reap matches claude-hooks. For opencoti, sessions are typically longer; revisit if cold-start latency turns out to be an issue in production use.
• Sqlite-vec dep choice. The Bun-native binding vs better-sqlite3 + the sqlite-vec extension. Decide at M5 based on which builds cleanly across Linux/macOS/Windows.

Risks

• F4 M3 patch surface area. Slot-init touches code that F5's ReST-KV (M1) and HeadInfer (M2) also modify. Ordering: F4 M3 lands first as patch 0001-; F5's start at 0010-. On a later upstream pin bump, M3 needs to rebase first.
• Asset hosting. While we pin claude-hooks' release, the embedder's lifecycle is tied to claude-hooks' tagging cadence. Mitigation: mirror to opencoti's own release once it's set up.
• @opencoti/tiers ↔ @opencoti/embedder boundary. The embedder depends on tiers (for GPU detect + download helper), and tiers dynamic-imports @opencoti/llamafile. No cycle introduced (embedder is the new edge of the DAG), but confirm with bun --cwd packages/opencoti-embedder typecheck after the shared-download refactor.