c4: polykv_api doc

docs/features/polykv_api.md

@@ -0,0 +1,573 @@
+# PolyKV Control-Plane API — Design (F5 M6-S5 / #712)
+
+**Status:** design (2026-07-16). Implementation is phased (§10); nothing here
+is shipped yet.
+**Inputs:** [polykv_api_research.md](polykv_api_research.md) (design-space
+survey), [poly_kv.md](poly_kv.md) (the KV mechanism this sits on top of),
+[llamafile-usage.md](../llamafile-usage.md) §3.4 (SharedKVPool user surface).
+**Single rule:** additive / opt-in. Every endpoint and field lives under the
+existing `opencoti` JSON namespace or new `/polykv/*` routes; no upstream
+behavior changes when PolyKV control is unused.
+
+---
+
+## 1. Goals
+
+A rich, opt-in control plane that satisfies every multi-agent serving journey
+on top of the existing SharedKVPool (`shared_pool_slot` +
+`shared_prefix_n_tokens`, server-context.cpp:3085). Four requirements, plus
+two the requirements imply:
+
+1. **Full status** of pools + the sessions attached to each.
+2. **tps reporting** — aggregate system throughput *and* per-session
+   generation tps. (All-new: upstream llama.cpp reports per-*response*
+   `timings` but has no per-session/slot running instrumentation.)
+3. **Autonomous capacity/admission loop** — a main session sets a per-session
+   gen-tps target (e.g. 5 tps); when the average across active sessions
+   approaches the floor the orchestrator holds off spawning sub-agents.
+   Behavioral knobs: reject a new session vs. warn.
+4. **LangGraph extensions** — a custom `BaseChatModel`, the pool-control API
+   exposed as LangGraph tools, and PolyKV-aware agent-state routing.
+5. **Nesting** (implied by 3): a sub-agent that discovers fan-out work can
+   become a pool parent for its own sub-agents, so parallelizable tasks run
+   in parallel instead of being forced sequential.
+6. **Fork / branch** (COW): a sub-agent group that shares an *ancestor* prefix
+   but then diverges (different toolset/context) gets its own shared pool
+   branched at the divergence point.
+
+## 2. Decisions locked (2026-07-16)
+
+| # | Decision | Choice |
+|---|----------|--------|
+| API home | where the control plane lives | **Self-contained in the C++ engine** — extends `/props`, `/slots`, adds `/polykv/*`. Works standalone; no `opencoti-server` in the path. |
+| Admission | enforce the tps floor or advise | **Both** — advisory (report) by default; optional per-pool enforced mode with `reject`\|`warn`\|`queue`. |
+| Pool model | how pools are created | **Explicit first-class objects** — `create`/`fork`/`attach`/`release`, immutable from birth (§3.3). Contiguous-prefix contract checked at fork. Boilerplate hidden by the opencoti plugin. |
+| Scope | deliverable shape | **Design the full surface now; implement in gated phases** (§10). |
+| Fork | divergent child prefix | **First-class in v1** (copy-free COW, §3.2 / §7) — *not* deferred. |
+
+**Defaulted (override any):** transport = HTTP on the engine + SSE for the
+live tps stream (poll fallback); tenancy = single trust domain (no per-pool
+ACL); tps = per-session *experienced* decode tps (EWMA) + aggregate decode
+throughput, floor = mean of active sessions' experienced tps (prefill
+excluded); nesting depth uncapped; interior pools pinned while any descendant
+is attached + grace TTL; seq-ids for pools+leaves sized at boot.
+
+---
+
+## 3. Data model
+
+### 3.1 A pool is a tree node
+
+```
+Pool {
+  id            : int32          // seq_id in the unified cache
+  parent        : int32 | ROOT   // -1 for a root pool
+  branch_pos D  : int32          // shared-ancestor length inherited from parent
+  own_range     : [D, L)         // tokens this pool materialized itself
+  prefix_len L  : int32          // total shared prefix = D + |own_range|
+  tokens        : [llama_token]  // full prefix token array [0, L), host-side (auto-P + fork validation)
+  prefix_hash   : [u64]          // cumulative rolling hash per position (validates any D; ~8 B/token host RAM)
+  pin           : bool           // resident-pinned (auto while descendants attached)
+  children      : [Pool]         // interior pools branched/extended from this one
+  sessions      : [Session]      // leaf sharers attached at this pool's prefix
+  last_access   : ts
+}
+Session {                        // a leaf = an ordinary decoding slot
+  id            : int32          // seq_id
+  pool          : int32          // the pool it attached to
+  suffix_range  : [L, L+S)       // its own private, mutable continuation
+  tps_ewma      : f32            // experienced decode tps
+  ttft_ms, state, ...
+}
+```
+
+The **root** pool (`parent = ROOT`, `D = 0`) is what today's
+`shared_pool_slot` request already produces. Everything else is new.
+
+### 3.2 Three edge types, one mechanism
+
+A child pool inherits `[0, D)` from its parent and owns `[D, L)`. `D` selects
+the edge type:
+
+- **extend** — `D = parent.L`. Child = parent prefix **+** appended shared
+  tokens. This is *nesting*: a sub-agent adds its own shared context and hands
+  it to its sub-agents.
+- **branch (COW fork)** — `D < parent.L`. Child shares the common ancestor
+  `[0, D)` and diverges with its own `[D, L)`. This is the fork: same system
+  prefix, different toolset after `D`.
+- **root** — `parent = ROOT`, `D = 0`.
+
+**Why the fork is copy-free.** `seq_cp` in the unified cache is additive
+membership: for each cell where `seq_has(cell, parent)` in `[0, D)` it calls
+`seq_add(cell, child)` (llama-kv-cache.cpp:2007-2044). No cell is duplicated —
+the ancestor cells simply gain the child in their sequence set. Because
+prefixes are **immutable** (FROZEN, §3.3), the shared range is never mutated,
+so the "copy" a classic COW would eventually pay never happens. The child only
+computes new KV for `[D, L)`, prefilled at RoPE position `D` attending to the
+shared `[0, D)` already in cache — exactly how attaching with a shorter
+`shared_prefix_n_tokens` works today. **Extend and branch are the same code
+path parameterized by `D`.**
+
+### 3.3 Immutability & versioning
+
+**Pools are immutable from birth.** A pool's prefix is fully materialized at
+`create`/`fork` and never changes afterwards — there is no OPEN/extend-in-place
+state. "Extending" a pool *always* means creating a child (`fork` with
+`D = parent.L`). This is simpler than a freeze-on-first-attach state machine
+and loses nothing: the "build context incrementally" journey is covered by the
+main agent decoding in its **own session** and then forking `from_session`
+(§4.1), which snapshots its current context as a new pool in one call — no
+in-place mutation ever needed.
+
+This makes "add a tool to the shared prefix after agents attached"
+well-defined — it is a *new versioned pool* (a fork), and the old one keeps
+serving its sharers unchanged. (This is the precise answer to the HuggingFace
+question: the prefix is static per pool; a tool/MCP added later is a new pool
+version, valid only for sessions that attach to it — never mutated under
+sessions already on the old one.)
+
+### 3.4 Lifetime
+
+Cell freeing is implicit-refcounted: a cell is reclaimed only when its sequence
+set empties, so an ancestor's cells physically survive as long as *any*
+descendant references them — even if the ancestor's own slot is torn down. On
+top of that:
+
+- **Pin policy.** Interior pools are auto-pinned (`--no-clear-idle` semantics,
+  per-node) while any descendant is attached. Without this a new leaf attaching
+  to an idle-cleared interior pool hits the residency guard
+  (server-context.cpp:3116) and silently falls back to full reprocess —
+  correct but slow. A **grace TTL** delays reclaim after the last descendant
+  detaches (cheap re-attach for bursty fan-out).
+- **seq-id budget.** Every pool *and* every leaf consumes one `n_seq_max` slot.
+  A deep/wide tree spends seq-ids on interior nodes too; the boot-time reserve
+  accounts for both. Exposed in `/capacity`.
+
+### 3.5 Materialization — the slot/seq decoupling (P2's real work)
+
+Today a "pool" is just an ordinary server **slot** that happened to serve a
+request: `shared_pool_slot` names a slot index, the pool's tokens live in
+`slots[pool].prompt.tokens`, and the pool seq **is** the slot's seq. First-class
+pools break that coupling, and this is the largest single implementation lift
+in the milestone:
+
+- **seq-ids beyond slots.** Pool seq-ids come from a boot-time reserve above
+  the slot range: `n_seq_max = n_parallel + --polykv-max-pools` (default
+  proposed: 16). A pool occupies a seq-id but **no slot** once materialized.
+- **Prefill-only internal task.** `create`/`fork` materializes `[D, L)` via an
+  internal task that runs through the normal batching path (so it composes
+  with every KV feature) but emits **no logits and no generation**; it
+  transiently borrows a slot for the prefill, then the KV stays on the pool's
+  seq and the slot is returned. `seq_cp(parent → pool, 0, D)` runs first for
+  fork/extend.
+- **Host-side pool state.** The Pool object (tokens, cumulative hash, tree
+  links, pin/TTL) lives in server memory beside the slot table — *not* in a
+  slot — and is what `/pools` reports and attach validates against.
+- **Unified KV required.** The split-stream `seq_cp` overload hard-asserts
+  `is_full` (llama-kv-cache.cpp:2063); pool sharing is only defined under
+  `--kv-unified` — already the SharedKVPool prerequisite, now stated as a
+  boot-time check when `--polykv-max-pools > 0`.
+
+---
+
+## 4. API surface
+
+All additive. Reuses the `opencoti` JSON namespace (`get_opencoti_props()`
+:3787) and the `/slots` accumulator precedent (:183). Endpoint shapes follow
+research §6.
+
+### 4.1 Pool lifecycle (imperative, explicit)
+
+| Verb | Endpoint | Body → result |
+|------|----------|---------------|
+| create root | `POST /polykv/pools` | `{tokens? \| from_session?}` → `{pool_id, prefix_len}`. `from_session` snapshots a live session's current context as the pool prefix — the **primary** creation path for agents (no token round-trip). |
+| fork (extend or COW branch) | `POST /polykv/pools/{id}/fork` | `{branch_pos D, tokens[D:L] \| from_session?}` → new child `{pool_id, parent:id, branch_pos, prefix_len}`. `D == parent.L` ⇒ extend (nesting); `D < parent.L` ⇒ COW branch. |
+| attach | *(request path, unchanged mechanism)* | per-request `pool_id` (legacy `shared_pool_slot` kept). **`shared_prefix_n_tokens` becomes optional**: the server auto-computes `P` as the longest hash-match between the request prompt and the pool's stored tokens (**auto-P**) — killing the mis-set-P footgun and the HF-question confusion. An explicit `P` is still accepted and validated. |
+| pin/evict/flush | `POST /polykv/pools/{id}/{pin\|evict\|flush}` | retention control (research §6 Req 1) |
+| release | `DELETE /polykv/pools/{id}` | detach + reclaim when refcount hits 0 (respects grace TTL) |
+
+Contiguous-prefix contract is checked at `fork`: the child's declared
+`tokens[0:D]` must hash-match the parent's `[0:D)` (cumulative `prefix_hash`
+at `D`), else `409 Conflict` — the caller then either re-roots or
+full-prefills. All routes live under a `/polykv/` prefix so a future upstream
+llamafile/llama.cpp bump can never collide with an unprefixed `/pools`.
+
+### 4.2 Status (research §6 Req 1)
+
+- `GET /polykv/pools` → `{pools:[{pool_id, parent, branch_pos, prefix_len,
+  own_kv_bytes, subtree_kv_bytes, residency, pinned, last_access_ts,
+  children:[pool_id], sessions:[{session_id, suffix_n_tokens,
+  marginal_kv_bytes, tps_ewma, ttft_ms, state}]}], tree_depth, seq_ids_used,
+  seq_ids_max}`. The `parent`/`children` fields make the **tree** explicit.
+  **KV attribution is by ownership, never by membership**: a shared cell
+  belongs to many seqs, so summing per-seq bytes would double-count — a pool
+  reports `own_kv_bytes` = its `[D, L)` range only, plus a `subtree_kv_bytes`
+  rollup (own + all descendants + attached sessions' suffixes). Σ own over the
+  tree == physical bytes, by construction.
+- `GET /polykv/pools/{id}` → one subtree.
+- `/slots` gains `pool_id` + `tps_ewma` per slot (extends #677).
+- `/props` `opencoti` block gains a `polykv` sub-object (config: floor,
+  admission mode, pin/TTL policy).
+
+### 4.3 tps (research §6 Req 2)
+
+- **Per-response** (already partly present): `timings{predicted_per_second,
+  prompt_per_second, cache_n}`.
+- **Headers** (TGI model): `x-session-tps`, `x-cached-prefix-tokens`,
+  `x-pool-id`. **Deferred to P2+**: HTTP headers must be emitted before the
+  body, but the slot (and thus the live tps value) is assigned only after the
+  task is queued — a header set at request time would always be stale, and
+  trailers aren't portable. The per-response `timings` JSON already carries the
+  authoritative per-response rate; headers add value only once pool attach
+  happens at request routing (P2), where `x-pool-id`/`x-cached-prefix-tokens`
+  are known up front.
+- **Live stream:** `GET /polykv/tps` (SSE) — periodic
+  `{ts, aggregate_tps, sessions:[{session_id, pool_id, tps_ewma}], mean_active_tps}`.
+  This is the signal the admission loop subscribes to.
+- **`/metrics`** (Prometheus, low-cardinality per research §4): gauges
+  `opencoti_polykv:gen_throughput_tps{pool}`, `pool_kv_bytes{pool}`,
+  `pool_sessions{pool}`; counters `prefix_cache_hits_total{pool}`,
+  `queries_total{pool}`. Label = pool only (never session — cardinality).
+
+### 4.4 Admission (research §6 Req 3; decision: both modes)
+
+- `POST /polykv/pools/{id}/admission` `{target_tps_per_session, mode:"advisory"|
+  "enforced", on_saturation:"reject"|"warn"|"queue"}` — per-pool policy.
+- `GET /polykv/pools/{id}/capacity` →
+  `{can_admit, projected_mean_tps_if_admitted, headroom_sessions, kv_headroom_pct, reason}`.
+  `headroom_sessions` is computed against **marginal** per-session KV
+  (`S_i·bpt`), not full context — the basis of the ~7× reduction (research §5).
+- **Advisory (default):** endpoints report; the orchestrator decides. Nothing
+  is rejected server-side.
+- **Enforced (opt-in):** on a new attach with projected `mean_active_tps <
+  floor` OR `kv_headroom` exhausted → `reject` = `429`/`503` + `Retry-After`
+  (RFC 6585/9110); `warn` = admit + `X-Sessions-Remaining` header; `queue` =
+  hold until headroom returns.
+- **Throttle-toward-floor is explicitly NOT v1.** Andes' insight (over-floor
+  generation is wasted GPU → throttle fast sessions back toward the floor to
+  reclaim capacity) requires per-slot rate control inside the continuous-
+  batching decode loop — i.e. skipping a slot in some decode batches. That is
+  scheduler surgery with its own correctness/perf gates, not an API feature;
+  it is banked as a P4-follow candidate, and v1 `warn` is headers-only.
+
+---
+
+## 5. tps instrumentation (all-new, C++ server-side)
+
+Upstream has no per-session running tps. Design:
+
+- Extend the `/slots` lifetime accumulators (server-context.cpp:183) with a
+  per-slot decode-token counter + wall-clock, updated at each decode-batch
+  completion for the tokens attributable to that slot.
+- **Experienced tps** per session = EWMA over a rolling window (default: last
+  `W` decoded tokens or `T` ms, whichever first) of that slot's decode
+  throughput. This is the quantity that *falls as concurrency rises* even while
+  aggregate climbs (attention doesn't batch-amortize — research §5), so it is
+  the correct SLO signal.
+- **Aggregate** = system decode throughput (all slots' decode tokens / wall).
+- **`mean_active_tps`** = mean of experienced tps across currently-decoding
+  sessions — the value the floor compares against (matches "average sessions
+  gen tps").
+- Prefill tokens are excluded from "gen tps" (counted separately as
+  `prompt_per_second`).
+- Cost: a handful of integer counters + one EWMA update per slot per batch —
+  negligible; gated behind the PolyKV block so off-path is byte-identical.
+
+---
+
+## 6. Admission control
+
+Two coupled signals (research §5):
+
+1. **Soft / low-lag:** `mean_active_tps` vs the floor. Directly measures the
+   SLO; leads `num_requests_waiting` (which lags).
+2. **Hard wall:** pool KV-occupancy %. Reject *before* this forces
+   preempt-oldest eviction.
+
+`projected_mean_tps_if_admitted` estimates the post-admit floor from the
+`B_sat` knee model (below the knee admitting is ~free; above it every admit
+costs everyone's TPOT ~linearly). On high-VRAM cards (RTX 6000 96 GB) the tps
+knee is hit **before** the KV wall, so signal (1) dominates on opencoti's
+hardware — which is exactly why the floor, not occupancy, is the headline.
+
+Framing: "5 tok/s/session" = **TPOT SLO 200 ms/token; maximize goodput s.t.
+TPOT ≥ floor** (a floor, not a maximize — Andes). The default is advisory: the
+LangGraph capacity-gate node (§8) reads `/capacity` and decides. Enforced mode
+is the same math with server-side `reject`/`warn`/`queue`.
+
+---
+
+## 7. Nesting & COW-fork — worked journeys
+
+**Journey A — sub-agent parallelization (nesting / extend).** Main agent M
+runs on root pool `P0 = [system+tools]` (len N0, FROZEN). M builds task context
+and `POST /pools/P0/fork {branch_pos:N0, tokens:[N0:N1]}` → interior pool `P1 =
+[system+tools+task]`. M spawns k sub-agents that each attach to `P1` and decode
+their own suffix in parallel. Physically, `P1`'s `[0:N0)` cells are P0's cells
+(now in seq-set {P0,P1,leaf…}); `[N0:N1)` is prefilled once. The k sub-agents
+cost only their marginal suffixes. The fan-out work that used to run
+sequentially now runs concurrently under one shared branch.
+
+**Journey B — divergent toolset (COW fork).** Two sub-agent groups share the
+system preamble `[0:D)` but need different toolsets. Group B does
+`POST /pools/P0/fork {branch_pos:D, tokens:[D:D']}` (D < N0) → `P1' =
+[system+toolsetB]`, sharing `[0:D)` zero-copy, prefilling `[D:D')` fresh. Its
+sub-agents attach to `P1'`. No physical copy of the shared preamble.
+
+**Invariants enforced.** (a) contiguous-prefix hash check at `fork`; (b)
+immutability from birth — a pool's prefix is fully materialized before any
+descendant can attach (§3.3), so no attach ever races a prefix change; (c)
+recursive pin — interior pools stay resident while descendants attached; (d)
+seq-id budget counts interior nodes. Divergence that is *not* a clean prefix of any
+existing pool → new root (or full prefill) — reported, never silently wrong.
+
+**Hybrid/recurrent guard (bug-2203) is recursive.** Every attach/fork on a
+recurrent or hybrid model still hits the exact-full-state guard
+(server-context.cpp:3116) at every level.
+
+---
+
+## 8. LangGraph integration (research §6 Req 4)
+
+Package home: a new **top-level `opencoti/` non-bun root** (per the repo's
+"new non-bun roots" rule) — Python package `opencoti-langgraph` (pip). Kept out
+of the bun workspace. Pin `BaseChatModel` signatures to a specific
+`langchain-core` version tag (the API is mid-migration — research caveat).
+
+- **`OpencotiChatModel(BaseChatModel)`** — implement `_generate` + `_llm_type`
+  (required), `_stream`/`_agenerate`/`_astream`; override `bind_tools`.
+  Populate `usage_metadata` and
+  `response_metadata{pool_id, session_tps, backpressure}` from the headers/§4.3.
+- **Pool-control tools** — `create_pool`, `fork_pool`, `pool_status`,
+  `session_tps`, `capacity` exposed as LangChain tools via `bind_tools` /
+  `ToolNode`, so an agent can manage its own pool tree.
+- **Capacity-gate node** — reads `/capacity`; returns
+  `Command(update={"capacity_ok":ok, "headroom_sessions":n}, goto=...)` to
+  route the graph (spawn vs. hold). `interrupt()` for block-until-free.
+  `InMemoryRateLimiter` is explicitly insufficient — it can't see per-session
+  tps or KV headroom.
+- **Agent-state routing** — a reducer surfaces `headroom_sessions` /
+  `mean_active_tps` into graph state so conditional edges branch on live PolyKV
+  capacity.
+
+---
+
+## 9. Composition with the KV stack
+
+PolyKV control is orthogonal to the KV *mechanics* and must not perturb them:
+
+- **rolling-KV / DCA / turbo / TCQ / quant-KV** — pool sharing is `seq_cp`
+  membership on whatever cell type is configured; the tps/admission plane is
+  read-only over the existing decode loop. Off-path byte-identical (gate).
+- **MTP / speculative** — per-session tps counts *accepted* tokens (the user-
+  visible generation rate), not draft steps.
+- **`--parallel ≥ 2` / `--kv-unified`** — the multi-slot requirement is already
+  the SharedKVPool baseline; the tree just adds interior seq-ids.
+- **hybrid/recurrent** — bug-2203 guard, recursive (§7).
+
+## 10. Phased implementation plan (gated)
+
+Full surface designed here; implementation lands in gated phases, each with a
+correctness + off-path-identity gate before the next:
+
+- **P1 — tps instrumentation** (§5). Per-slot counters + EWMA + `/slots`/`/props`
+  exposure + `/polykv/tps` SSE + `/metrics`. Prereq for everything.
+  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
+  gates green — sustained EWMA −3.2% vs `timings` ground truth; 2-session
+  concurrency shows correct `aggregate_tps` ≈ Σ sessions + `mean_active_tps`;
+  `/metrics` 3 series render. bug-2205: aggregate MUST come from
+  Σ `n_decoded_cum` (per-token), not `n_tokens_predicted_total`
+  (completion-time only).
+- **P2 — pool objects + status API** (§3, §4.1-4.2). The slot/seq decoupling
+  (§3.5): seq-id reserve, prefill-only materialization task, host-side Pool
+  state, **auto-P attach**; `GET /polykv/pools`. Root pools only.
+  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
+  gate 23/23 on Qwen3-4B — boot check (`--polykv-max-pools` without
+  `--kv-unified` fails clean; NOTE `--parallel` must be explicit, unset
+  auto-sets `kv_unified=true`), create-from-prompt prefill-only P=95,
+  auto-P exact attach `cache_n=95`, divergence attach `cache_n=42<95`,
+  `from_session` zero-copy `seq_cp` snapshot (affinity is evict-on-reuse —
+  only the most-recent session per slot resolves), pin/unpin/GET-one,
+  release + released-id clean fallback, disabled path inert, pooled vs
+  unpooled greedy text IDENTICAL (bit-shared cells). Mutations only on the
+  server thread via `SERVER_TASK_TYPE_POLYKV`; recurrent guard per bug-2203.
+- **P3 — nesting & COW-fork** (§7). `fork` with arbitrary `D` (extend + COW
+  branch), cumulative-hash contract check, recursive pin, seq-id accounting.
+  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
+  gate 18/18 substantive on Qwen3-4B — extend fork (D=parent.L, own=22),
+  COW branch (D=47<95, zero-copy `seq_cp` of `[0,D)` + fresh prefill),
+  contract violation → clean 4xx, auto-P attach to a child (cache_n=117),
+  tree reporting (parent/children/branch_pos/own_len/tree_depth), release
+  ordering enforced (parent-with-children refused, leaves-first drains),
+  seq exhaustion at max_pools, `from_session` fork. The §4.1 prefix
+  contract is enforced **token-exact** (`std::equal` against the parent's
+  stored tokens — strictly subsumes the cumulative-hash check; `prefix_hash`
+  kept as identity fingerprint). Pooled-vs-unpooled greedy divergence on the
+  branch was classified benign by first-token probe (same top-5 ranking,
+  max |Δlogprob| 0.112 — batch-shape numerics, #495 class). Recurrent
+  bug-2203 guard is recursive: hybrid/recurrent forks must extend at
+  D == parent.L.
+- **P4 — admission** (§6). `/capacity`, advisory metrics, then enforced mode
+  (`reject`/`warn`/`queue`).
+  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending;
+  `queue` deferred per §11 default): gate 18/18 on Qwen3-4B — /capacity shape
+  (idle: can_admit, projected null, kv cells by OWNERSHIP: pools' own ranges +
+  slots' marginal suffixes via new `slot.n_pool_shared`), advisory floor never
+  rejects, **enforced+reject → 429 + Retry-After** under an active decoding
+  session (tps-floor arm needs n_active ≥ 1), **enforced+warn → 200 +
+  X-Sessions-Remaining**, off-path (no pool_id / unknown pool_id) untouched.
+  Projection = conservative fully-saturated knee: `mean_active · n/(n+1)`
+  (never over-admits below the real B_sat knee). Fast path: an atomic
+  enforced-pool count — ordinary traffic pays zero; the gate itself is a
+  high-priority OP_CAPACITY control task from `handle_completions_impl`
+  (same pattern as /slots' METRICS task).
+- **P5 — LangGraph package** (§8). `opencoti-langgraph` under top-level
+  `opencoti/`. Runs in the first-class **`opencoti` conda env** (py3.11,
+  langchain-core 1.4.x + langgraph 1.2.x pinned; the same env will later
+  host the consultants council alongside opencoti-server). Includes the P6d
+  `CompactionNode` (§13).
+  **P5 ✅ IMPLEMENTED 2026-07-16**: offline 21/21 (pytest, httpx MockTransport
+  FakeServer), live gate **19/19** on Qwen3-4B `:8231 --parallel 2 --kv-unified
+  --polykv-max-pools 4` — pool tree (token-path fork D=65), `OpencotiChatModel`
+  invoke + streaming with pool attach proven (`cache_n=68 ≥ root prefix 65`),
+  `session_tps`/usage metadata, capacity context arm + gate-node routing,
+  enforced 429 → `PoolSaturatedError(retry_after)` → advisory recovery,
+  **P6c compact-by-refork e2e** (old pool released, migrated session coherent
+  on the new pool, `cache_n=71`), tps `ctx_*` fields. Contract lessons
+  (bug-2208): fork `prompt`/`tokens` = the child's **FULL prefix [0,L)**
+  token-exact vs the parent (never suffix-only — build via `/tokenize`:
+  parent tokens + suffix `add_special=false`); a pool prefix meant for
+  `/v1/chat/completions` attach must be the **chat-templated** system block
+  (raw text shares 0 tokens → `cache_n=0`); Qwen3 thinking models need
+  `chat_template_kwargs: {enable_thinking: false}` for content-based gates.
+  Client accordingly ships `tokenize()` + `compact_by_refork(compacted_tokens=…)`.
+- **P6 — context exhaustion & compaction** (§13). **P6a (ctx-shift pool
+  guard) + P6b (context visibility + /capacity context arm) land BEFORE P5**
+  per user decision 2026-07-16; P6c/P6d ship with P5.
+  **P6a+P6b ✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture
+  pending): gate 23/23 on Qwen3-4B at `-c 4096 --parallel 1 --context-shift`
+  (the exact §13.1-regime-3 configuration) — boot WARN; pooled session forced
+  past n_ctx = graceful `truncated=true` stop at 4021 tokens (position wall
+  fires one token before unified-cell exhaustion when the prefix is
+  pool-shared), NO shift, pool provably coherent afterwards (fresh attach
+  cache_n=60, needle answered); non-pooled ctx-shift still engages and
+  completes after pools released (off-path intact); `?expected_tokens` context
+  arm rejects with `"context headroom exhausted"`; `compaction_pressure`
+  0 at idle, ∈[0,1]; `orphaned_pin` true for a pinned never-attached leaf;
+  `/polykv/tps` sessions carry `ctx_used/ctx_total/ctx_headroom_tokens`.
+  Guard bookkeeping: `slot.n_pool_shared` is clamped to the retained prefix at
+  prompt-cache decision time (`min(n_pool_shared, n_past)`) — NOT blind-reset,
+  because cache reuse keeps pool-shared cells across later non-pool requests
+  on the same session. NOTE the geometry lesson: with `--kv-unified` the slot
+  n_ctx is the FULL `-c` (not divided by `--parallel`), and multi-slot cell
+  exhaustion precedes the positional wall — gates must use `--parallel 1`.
+
+## 11. Open sub-decisions (defaulted; flag to change)
+
+- EWMA window `W`/`T` defaults (proposed: 128 tokens or 2 s).
+- Grace TTL for interior-pool reclaim (proposed: reuse `--slot-shrink-idle-ms`).
+- `/metrics` opt-in flag name + default-off.
+- Whether `queue` (admission) is v1 or P4-follow (proposed: `reject`+`warn` v1,
+  `queue` follow).
+- `--polykv-max-pools` default (proposed: 16; each reserved seq-id costs KV
+  bookkeeping but no cells until materialized).
+- Hash algorithm for the cumulative `prefix_hash` (proposed: xxhash64 rolling;
+  8 B/token host RAM — ~800 KB for a 100k-token prefix, negligible).
+- SSE tps-stream cadence (proposed: 500 ms, configurable; stream is opt-in so
+  off-path cost is zero).
+
+## 12. Verification
+
+- Off-path identity: PolyKV-control-unused boot is byte-identical (DSO + decode).
+- tps accuracy: instrumented per-session tps vs. an external wall-clock
+  token-rate measurement, ±few %.
+- Fork correctness: a branched pool's decode is logit-equivalent to the same
+  session full-prefilled (real_frac=0 / KLD), never greedy-needle.
+- Tree lifetime: ancestor cells survive descendant-only references; reclaim on
+  refcount 0; recursive pin prevents the residency-miss fallback.
+- Admission: projected vs. realized `mean_active_tps` under a spawn ramp;
+  `reject`/`warn` behavior + `Retry-After`/`X-Sessions-Remaining` headers.
+- Composition gates: rolling-KV / DCA / turbo / MTP / `--parallel 2` unchanged.
+
+## 13. Context exhaustion & compaction orchestration (P6)
+
+*Added 2026-07-16 on user review: the design above admits sessions by tps floor
+and marginal KV, but had no story for what happens when the shared context
+actually fills — and continuous agentic frameworks WILL fill it. Decision:
+P6a+P6b land BEFORE P5 (the LangGraph capacity gate builds on the context arm);
+P6c/P6d ship with/after P5.*
+
+### 13.1 What the engine does today (verified, three regimes)
+
+1. **Prompt too long at admission** → clean `ERROR_TYPE_EXCEED_CONTEXT_SIZE`.
+2. **Generation reaches `n_ctx`, `--ctx-shift` off (default)** → clean stop
+   with `truncated=true` (`server-context.cpp:1898`). Not corrupting, but
+   silent mid-thought truncation is exactly the "fails badly" mode for agents:
+   no advance warning, no orchestration signal.
+3. **`--ctx-shift` on + pool-attached session → CORRUPTION.** The shift does
+   `seq_rm` + `seq_add(…, -n_discard)` (`server-context.cpp:3261-3262`); in
+   the unified cache `seq_add` mutates the **per-cell position**, and
+   pool-shared cells are members of many seqs — the shift re-RoPEs the pool's
+   prefix for every other session and every descendant pool. Upstream guards
+   this for parent/child shared prompts (`server-context.cpp:3192`) but a
+   pool attach is neither, so the guard does not fire. The rest-kv-eviction
+   variant uses the same primitives and has the same hazard.
+
+The "safe" naive alternative — COW-copy shared cells before shifting —
+allocates a duplicate of the prefix at the exact moment the cache is full.
+Both intuited failure modes (corruption / usage inflation) are real.
+
+### 13.2 Design principle: compaction is a prompt REWRITE, never an in-place KV op
+
+LangGraph (and every agentic framework) compacts at the prompt-assembly
+level: `trim_messages`, `RemoveMessage` reducers, or a summarization
+`pre_model_hook` that replaces old messages with a running summary. The
+compacted context is **new text** — its tokens cannot match the old cells, so
+cell-level "compaction" is not even meaningful. Therefore the server must
+never mutate pooled cells; the correct primitive already exists in the tree:
+
+> **Compaction = re-root via fork.** The system prompt + tool definitions
+> survive compaction verbatim — that is the stable ancestor pool
+> `[0, D_sys)`. Compacted context = `fork(ancestor, D=D_sys,
+> tokens = summary + recent tail)` — a new sibling branch, prefilled once.
+> Sessions migrate to the new pool; the old working subtree is `release`d.
+> Pool immutability is preserved by construction, no cell position is ever
+> touched, and net cells go **down** (old subtree reclaimed) instead of up
+> (COW duplication).
+
+Pinned pools under this model are a **leak hazard, not a corruption hazard**:
+a pin left on the abandoned subtree blocks reclaim. Discipline is
+unpin-after-migrate; the server reports orphans (pinned + zero sessions +
+zero children) so the orchestrator can sweep.
+
+### 13.3 Phases
+
+- **P6a — corruption guard (before P5).** Extend the ctx-shift refusal to
+  pool-attached slots (`slot.n_pool_shared > 0` ⇒ clean error, same as the
+  `:3192` shared-prompt refusal) + boot WARN when `--ctx-shift` is combined
+  with `--polykv-max-pools`. Rest-kv-eviction path included. Off-path inert.
+- **P6b — visibility + spawn-gate parity (before P5).** Per-session
+  `ctx_used`/`ctx_headroom_tokens` in `/polykv/tps` + `/slots`; `/capacity`
+  gains `?expected_tokens=N` and a **context arm** to `can_admit`
+  (reason `"context headroom exhausted"`) so sub-agent spawns are gated on
+  context exactly like the tps floor; plus a `compaction_pressure` field
+  (0..1, driven by free-cell fraction and largest-session share) so the
+  orchestrator compacts BEFORE the wall.
+- **P6c — compact-by-refork protocol (with P5).** Documented two-step:
+  summarize via normal completion → `fork` the stable ancestor at `D_sys`
+  with the compacted suffix → migrate sessions → `release` old subtree.
+  Orphaned-pin flag in `/polykv/pools`. Atomic convenience endpoint only if
+  the two-step proves racy in practice.
+- **P6d — LangGraph side (inside opencoti-langgraph).** `CompactionNode`
+  (pre_model_hook pattern: watch `response_metadata` headroom /
+  `compaction_pressure` → summarize → re-root → swap `pool_id` in graph
+  state); the capacity-gate node checks the context arm from day 1.
+
+### 13.4 Non-goals
+
+- In-place KV compaction/merging of pooled cells (meaningless under rewrite
+  semantics; corrupting under sharing).
+- Server-side self-summarization (the orchestrator owns the summary — it has
+  the conversation semantics; the server only has tokens).
+- Growing `n_ctx` at runtime (allocation is boot-time; capacity planning is
+  the admission plane's job).