# Protocol — Upstream Sync

opencoti is a soft fork of [opencode](https://opencode.ai). We sync
from upstream regularly. This document defines the rules that keep
syncs cheap.

## Upstream

- Tracked upstream: [mann1x/opencode](https://github.com/mann1x/opencode)
  (dev fork; opencode-of-record is anomalyco/opencode).
- Default upstream branch: `dev`.
- Sync cadence: weekly is the target, more often when upstream is
  hot.

## The rule

> **Every opencoti change is either an additive file outside the
> upstream tree, or a surgical hook inside it.**

### Additive (preferred)

A file that doesn't exist in upstream opencode. New files always live
in:

- `docs/`
- `vendors/sources/<tool>/`, `vendors/patches/<tool>/`
- `packages/opencoti-*/` (new bun workspace packages)
- `opencoti/` (top-level non-bun roots, e.g. Go for `opencoti-server`)
- Top-level identity files we own: `README.md`, `CONTRIBUTING.md`,
  `SECURITY.md`, `CLAUDE.md`, `STATE_SUMMARY.md`.

Additive files never conflict on upstream merges.

### Surgical hook

A small edit to an existing upstream file. **Allowed only when
there is no additive way to achieve the effect.**

A surgical hook MUST:

1. Be the minimum-line edit that achieves the effect — one import
   plus one call site, or one conditional.
2. Carry a marker comment on the same line or immediately above:

   ```ts
   // opencoti-hook: <feature-id> — see docs/features/<plan>.md
   ```

3. Defer to opencoti only when opencoti is enabled. With opencoti
   disabled (or absent), the upstream behavior must be byte-identical.
4. Be listed in the **Registry** below.
5. Have an entry in the relevant child plan's "Milestones" or
   "Surgical hooks" section.

### What is NOT a surgical hook

- A refactor of an upstream file "while we're here". File it
  upstream instead.
- A rename of an upstream symbol. Wrap, don't rename.
- A behavior change that is unconditional. Gate it behind an
  opencoti flag.

## Sync workflow

1. `git fetch upstream`
2. `git merge upstream/dev` (or rebase if the branch is local-only).
3. Resolve conflicts:
   - In additive files: there should be no conflicts. If there are,
     it means the file's path collided with new upstream content —
     rename our file.
   - In hook sites: keep the hook, accept upstream changes around
     it. The marker comment is the anchor.
4. Run the conformance check:
   `bun run scripts/check-hooks.ts` *(to be written — M0)*
   It greps for `opencoti-hook:` markers and verifies they match the
   Registry below.
5. Run typecheck and the test suites of every `packages/opencoti-*`
   package.
6. Sync STATE_SUMMARY.md (note the upstream SHA we synced to).

## Registry — known surgical hooks

Add an entry in the same PR you add the hook.

| Hook ID | File | Feature | Plan |
| --- | --- | --- | --- |
| `caliber-runtime` | `.gitignore` (trailing block) | Caliber pre-commit / session hooks runtime cache | This file (tooling, no per-feature plan) |
| `tiered-inference` (import) | `packages/opencode/src/session/llm.ts` (import block, just below `LLMRequestPrep` import) | F1 — Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
| `tiered-inference` (call site) | `packages/opencode/src/session/llm.ts` (inside `run` Effect.fn, just before the `flags.experimentalNativeLlm` seam) | F1 — Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
| `tiered-inference` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/tiers": "workspace:*"`) | F1 — Tiered Inference Engine | [features/tiered_inference.md](../features/tiered_inference.md) |
| `memory-tools-resolvability` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/memory": "workspace:*"`) | F4 M5 — Memory / Embedder | [features/memory_embedder.md](../features/memory_embedder.md) |
| `opencoti-default-plugins` (import) | `packages/opencode/src/config/config.ts` (just below the existing `ConfigPlugin` import) | F4 M5 follow-up — auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
| `opencoti-default-plugins` (call site) | `packages/opencode/src/config/config.ts` (in the config-loader Effect.fnUntraced body, just before the `return { config: result, … }`) | F4 M5 follow-up — auto-wire opencoti plugins | [features/memory_embedder.md](../features/memory_embedder.md) |
| `tiered-inference-setup` (import) | `packages/opencode/src/index.ts` (just after the existing `isRecord` import) | F1 M5.7 — Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
| `tiered-inference-setup` (call site) | `packages/opencode/src/index.ts` (`.command(tierCommand)` after `.command(DbCommand)`) | F1 M5.7 — Tier-0 setup CLI | [features/tiered_inference.md](../features/tiered_inference.md) |
| `opencoti-server-autostart` (import) | `packages/opencode/src/config/config.ts` (just below the `opencoti-default-plugins` import) | F3 M3 — opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
| `opencoti-server-autostart` (schema) | `packages/opencode/src/config/config.ts` (`opencoti: Schema.optional(Schema.Struct({ server: ... }))` block in the main config schema) | F3 M3 — opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
| `opencoti-server-autostart` (call site) | `packages/opencode/src/config/config.ts` (in the config-loader Effect.fnUntraced body, just after the `opencoti-default-plugins` call site, before `return { config: result, … }`) | F3 M3 — opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |
| `opencoti-server-client` (dep) | `packages/opencode/package.json` (`dependencies."@opencoti/opencoti-server-client": "workspace:*"`) | F3 M3 — opencoti-server TS client + autostart | [features/opencoti_server.md](../features/opencoti_server.md) |

The two TS edits carry `// opencoti-hook: tiered-inference — see docs/features/tiered_inference.md` markers; the `package.json` dep has no anchor comment (JSON has no comments) — this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the `run` Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.

**F3 M4 (2026-05-23): no new registry rows.** M4 (session +
turn events flowing into opencoti-server) ships as a pure
plugin (`@opencoti/session-events-plugin`) that subscribes to
opencode's typed `Hooks.event` firehose. Since the plugin API
already exposes session.created/updated/deleted/error/idle as
typed events, no surgical hook into upstream source is
required — matching the M5-D2 precedent. Hook count stays
at 9 source markers + 4 JSON deps from F3 M3. See
`docs/features/opencoti_server.md` "M4" for the design-pivot
rationale.

**F3 M5 (2026-05-23): no new registry rows.** M5
(tier-engine event sink) ships as a plugin
(`@opencoti/tier-events-plugin`) that injects a `Telemetry`
implementation into `@opencoti/tiers` via a new module-level
slot (`registered-telemetry.ts`, mirroring the existing
`active-config.ts` overlay pattern). The slot is consulted
by `runtime.ts:99` after the explicit hook arg and
test-injected exec opts, before `noopTelemetry` — adding the
sink without touching upstream source. The `Telemetry`
interface itself was added during F1 M5 with the docstring
already saying F3 would implement it, so M5 lands in a seam
that was waiting for it. Hook count stays at 9 source
markers + 4 JSON deps from F3 M3 (unchanged through M4 and
M5). See `docs/features/opencoti_server.md` "M5" for the
seam design.

**F3 M6 (2026-05-23): no new registry rows.** M6 (daemon
tool registry → opencoti sessions) ships as a plugin
(`@opencoti/server-tools-plugin`) that fetches the daemon's
`/v1/tools` catalog and injects a `Record<string, Tool>` into
`@opencoti/tiers` via a new module-level slot
(`registered-server-tools.ts`, mirroring M5's
`registered-telemetry.ts`). The slot is consulted by
`runtime.ts`'s `mergeTools` call **last** — after the
synthetic-tier and memory-bridge tool records that were
already merged there since F1/F4 M5 — so the daemon tools
land in a seam that already existed for exactly this kind of
record-merge, no upstream source touched. Hook count stays at
9 source markers + 4 JSON deps from F3 M3 (unchanged through
M4, M5, and M6). See `docs/features/opencoti_server.md` "M6"
for the design-pivot rationale.

**F3 M7 (2026-05-23): no new registry rows.** M7 (the pgvector
storage backend) is **entirely additive Go inside the
`opencoti/server/` root** — a new `internal/store/pgvector/`
package implementing the same `Store` interface family as
`internal/store/sqlitevec/`, plus a shared
`internal/store/storetest/` conformance harness and a pure-logic
`internal/store/common.go` both backends share. Backend selection
is a serve-time CLI flag (`--backend sqlite|pgvector`), not a build
flag, so no upstream file and no TS package is touched. Unlike
M4–M6, M7 ships **no plugin** — there is no opencode-side surface to
wire, since the backend is purely internal to the daemon. Hook count
stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through
M4, M5, M6, and M7). Note: M7 also moved the module + host Go
toolchain to 1.25 for the modern pgx/pgvector-go/testcontainers
releases — a build-environment change, not a sync-surface change.
See `docs/features/opencoti_server.md` "M7" for the backend design.

**F3 M8 (2026-05-23): no new registry rows.** M8 (multi-session
coordination — presence, broadcasts, advisory locks) is **additive Go**
inside `opencoti/server/`: a new in-memory `internal/coord` hub plus an
`internal/server/coord.go` HTTP surface (including the daemon's first
SSE endpoint), injected as a concrete dependency on `server.Options`.
The new `@opencoti/coordination-plugin` is the ecosystem's first SSE
consumer, but it wires in like M4–M6: a string appended to
`OPENCOTI_DEFAULT_PLUGINS` in opencoti-owned `@opencoti/tiers`, consumed
by the already-registered `opencoti-default-plugins` call site — **not**
a new source marker. The count stays at 9 source markers + 4 JSON deps
from F3 M3 (unchanged through M4–M8). No upstream file and no store
backend touched. See `docs/features/opencoti_server.md` "M8".

**F3 M9 (2026-05-23): no new registry rows.** M9 (opt-in shared
context) is **additive Go** inside `opencoti/server/`: a new
`internal/share` package (the share coordinator) plus an
`internal/server/share.go` HTTP surface, both reusing the existing
`Store` ACL methods (`SetSessionACL` / `ListCollections`) and the M8
`coord.Hub` — no new `Store` method, no schema change, no store backend
touched. The `@opencoti/coordination-plugin` is **extended in place** to
consume the two new `collection.shared` / `collection.unshared` bus
events and to add an opt-in `share_session_collection` flag; it remains
auto-wired through the already-registered `opencoti-default-plugins` call
site, so no new source marker. The count stays at 9 source markers + 4
JSON deps from F3 M3 (unchanged through M4–M9). See
`docs/features/opencoti_server.md` "M9".

**F3 M10 (2026-05-24): no new registry rows.** M10 (the sqlite↔pgvector
`migrate` subcommand) is **additive Go** inside `opencoti/server/`: a new
`MigrationStore` optional interface on `internal/store` plus a per-backend
`migrate.go` implementing it and a new `cmd/opencoti-server/migrate.go`
subcommand. Like M7 it has **no opencode-side surface** — no plugin, no
HTTP endpoint, no TS client method — so there is nothing to auto-wire and
no source marker to add. It touches no upstream file, no schema (dedupe
reuses existing columns/indexes), and no `go.mod`/`go.sum` (deps already
present; the new code is stdlib `encoding/binary` + already-vendored
drivers). The count stays at 9 source markers + 4 JSON deps from F3 M3
(unchanged through M4–M10). See `docs/features/opencoti_server.md` "M10".

**F5 M0–M3 (2026-05-24 → 2026-05-29): no new opencode source markers.**
The Advanced KV Cache series (`0006`–`0030`) lives entirely in the
**vendored** `vendors/sources/llamafile/` tree, captured into numbered
patches under `vendors/patches/llamafile/`. The 6 surgical hooks added
during F4 M5+M3 (`tiered-inference` x3, `memory-tools-resolvability`,
`opencoti-default-plugins` x2 — see rows above) stay unchanged. F5 M3
specifically also adds **two NEW non-upstream files inside the vendored
tree**: `vendors/sources/llamafile/llama.cpp/ggml/include/ggml-neo-pipeline.h`
and `.../ggml/src/ggml-neo-pipeline.cpp` (the NEO orchestrator). These
do **not** belong in this registry — they are not surgical hooks into
*upstream opencode*, they are additions to *vendored llama.cpp* (the
distinction in "What is NOT a surgical hook" above). They are tracked
in `vendors/patches/llamafile/README.md`'s 0030 row, which is the
canonical vendored-source registry.

**F5-opt W1–W2 (2026-05-30): still no new opencode source markers.** The
concurrency=1 optimization round (`0036`, `0040`) likewise lives entirely
in vendored `llama.cpp`. `0036` (W1, PCIe probe) adds **NEW** non-upstream
`common/pcie-profile.{h,cpp}`. `0040` (W2, wholesale iqk CPU-FA import) adds
the **NEW** vendored `ggml/src/iqk/**` engine tree (29 files), plus
`ggml/include/ggml-iqk-flash-attn.h` + `ggml/src/ggml-iqk-flash-attn.cpp`,
and carries one `opencoti-hook: f5-opt-cpufa` marker in the *vendored*
`ggml/src/ggml-cpu/ops.cpp` dispatch site — which, being under
`vendors/sources/`, is **not** counted by the `packages/ docs/` grep and is
registered solely via the `0040` README row. The `opencoti-llamafile`
adapter wires `--iqk-flash-attn` through plain config (no marker). Marker
count in `packages/ docs/` is therefore unchanged: **20** per
`git grep "opencoti-hook:" packages/ docs/ | wc -l` (this counter has read
20 since F4 M5+M3; earlier "18/19" figures in this file are stale prose,
not a real source delta). See `docs/features/advanced_kv.md` for the
per-milestone history.

**F5-opt W3–W4 + F5 M6 PolyKV/MTP (2026-05-30 → 2026-06-07): still no new opencode source
markers.** `0042` (W3, fused MoE up+gate), `0070` (W4, M7 Rolling KV), `0071` (cosmocc
state-IO, bug-269), `0072`/`0073` (M6 PolyKV — SharedKVPool + TurboQuant), and `0074`
(M6-S4 — speculative MTP draft head, #373/#404-410) all live entirely in vendored
`llama.cpp` (plus, for `0071`, the outer-vendor `llamafile/llamafile.{c,h}`, and for
`0073`, the outer-vendor `llamafile/build-functions.sh`) — **none** are hooks into upstream
*opencode* source, so none are counted by the `packages/ docs/` grep; each is registered
solely via its `README.md` row. `0074` touches only nested `llama.cpp/` files (24 files,
1 new: `src/models/gemma4-assistant.cpp`), no outer-vendor delta; its three surgical
upstream edits + the additive surface are detailed in the F5 M6-S4 block below.

> **Update 2026-06-16 (llamafile 0.10.3 bump):** `0074` — and the gemma-specific `0076` / `0077` — are now
> **RETIRED**, superseded by upstream-native NextN/MTP + gemma4 E-series support in `dbe9c0c` (the +613-commit
> jump under the 0.10.3 bump). See the per-patch RETIRED banners below and
> `docs/protocols/LLAMAFILE_UPGRADE.md`. The active stack is **35 patches**
> (`0001-0007, 0010, 0020, 0030-0036, 0040, 0042, 0070-0073, 0075, 0078, 0079, 0080, 0081, 0082, 0083, 0084, 0085, 0086, 0087, 0088, 0089`); the three retired entries are
> kept below for audit history only (their patch files no longer exist). **Re: `0079` —** the gemma assistant
> drafter (retired `0074`) is **restored on 0.10.3 as `0079-gemma-assistant-mtp.patch`** to *coexist* with
> upstream-native NextN as `--spec-type draft-assistant` (#423, a deliberate two-engine union): native NextN
> does **not** cover Gemma-4 (iSWA, no NextN head), so the assistant drafter is Gemma's only MTP path. See the
> `opencoti F5 M6-S4 mtp (0.10.3 restore)` block near the end of this file.

**`0070` marker count (S4 update, 2026-06-04):** the R2-a-era "6 markers" figure
is superseded. After the full Stage-3 position-window forward + the Stage-4
`--kv-residency-mode` knob landed (and S4-2 **removed** the throwaway iSWA-spike
+ ~29 env-gated diagnostic scaffolds), `0070` now carries **~40** `opencoti-hook:
f5-rolling-kv` / `rolling-kv M7` markers across `ggml/include/{ggml.h,ggml-iqk-flash-attn.h}`,
`ggml/src/{ggml-backend.cpp}`, `ggml/src/ggml-cpu/{ggml-cpu.c,ggml-cpu.cpp,ops.cpp,ops.h}`,
`ggml/src/ggml-cuda/{fattn.cu,fattn.cuh,fattn-common.cuh,ggml-cuda.cu}`,
`ggml/src/iqk/iqk_flash_attn.cpp`, `src/{llama-context.cpp,llama-graph.cpp}`
(the canonical inventory is `git grep -n "opencoti-hook:" vendors/sources/llamafile/llama.cpp`
= **54** total incl. the `f5-opt-fmoe`/`f5-opt-cpufa` markers from `0040`/`0042`).
The exact set is descriptive, not load-bearing: the `0070` README row is the
registry of record. **`0071`** carries no new in-tree marker — its 3-file edit
(the `llamafile_open_plain` def/decl + the `llama-mmap.cpp` COSMOCC fallback) is
registered via the `0071` README row. **Counter correction:** the
canonical opencode-source-hook figure is `git grep -h "opencoti-hook:" -- packages/`
= **10**, unchanged by W3/W4. The *combined* `packages/ docs/` counter now reads
**22** (the W1–W2 note above cites 20); that +2 is **pure docs prose** — this
file's W2 paragraph and `advanced_kv.md` both now mention `f5-opt-cpufa` by name —
not new source hooks. That combined counter has always conflated real markers with
documentation mentions; treat `packages/`-only = **10** as canonical, and the
vendored-patch markers as registered by their `README.md` rows.

**F5 M6 PolyKV S1 (2026-06-05): no new opencode source markers.** The
`opencoti-hook: poly-kv-pool` markers (`git grep -n "poly-kv-pool"
vendors/sources/llamafile/llama.cpp/tools/server` = **4**: 1 param block in
`server-task.h`, 2 parse lines in `server-task.cpp`, 1 gated prefill branch in
`server-context.cpp`) live entirely in vendored `llama.cpp` server source and are
additive + `shared_pool_slot < 0`-default-off (boot byte-identical). **Captured as
`vendors/patches/llamafile/0072-poly-kv-pool.patch` at M6-S3 (#372)** and registered
via that patch's `README.md` row — same mechanism as `0070`/`0071`. The
`packages/`-only canonical counter stays **10**. S1 runtime
prerequisite: the server must boot `--no-clear-idle` so the read-only pool slot's
KV stays device-resident for cross-seq `seq_cp` sharing (see `poly_kv.md` S1 / [[bug-367]]).

**F5 M6 PolyKV S2 perf-path Level A (2026-06-06): GPU-resident turbo dequant-on-lift.** New
`opencoti-hook: turboquant perf-lift` markers — `git grep -n "turboquant perf-lift"
vendors/sources/llamafile/llama.cpp` = **10** across **6 files**: 3 in new additive opencoti files
(`ggml/src/ggml-cuda/turbo-cpy.cu`, `turbo-cpy.cuh`, `turbo-innerq-dev.cuh` — file headers) and 7 in
**4 edited** vendored files (`ggml-cuda/set-rows.cu` ×2 = include + accessor impl; `ggml-cuda/cpy.cu`
×3 = 2 includes + 1 dispatch branch; `ggml-cuda/ggml-cuda.cu` ×1 = CPY `supports_op` accepting
`turbo{2,3,4,8}_0→F16`; `src/llama-graph.cpp` ×1 = single-cast `turbo→F16` lift). All additive +
default-inert (the CPY branch only fires for a `turbo→f16` cast, which only exists when `-ctk/-ctv
turboN` is set; f16/q8/q4 paths untouched, boot byte-identical). **Captured in
`vendors/patches/llamafile/0073-turboquant-kv.patch` at M6-S3 (#372)** and registered via that patch's
`README.md` row — same mechanism as `0070`/poly-kv-pool. The
`packages/`-only canonical counter stays **10**. See `poly_kv.md` S2 perf-path Level A.

**F5 M6 PolyKV S2 perf-path Level B (2026-06-06): fused FA-VEC turbo + new `GGML_OP_TURBO_WHT` op.**
New `opencoti-hook: turboquant perf-lift (… Level B)` markers, additive on top of Level A. **New ggml op
`GGML_OP_TURBO_WHT`** appended at the end of the op enum (after `FLASH_ATTN_TAIL_PARTIAL`, before
`GGML_OP_COUNT`) — same append-at-end discipline as `MOE_FUSED_UP_GATE`/`STREAMING_FLASH_ATTN`; the
host/DSO `GGML_OP_COUNT` (now **100**) and the name/symbol arrays + both static_asserts in `ggml.c` move
in lockstep, CPU forward is an abort-stub (CUDA-only op, like `STREAMING_FLASH_ATTN`). New additive files:
`ggml-cuda/turbo-wht.{cu,cuh}` + `turbo-wht-core.cuh`, `ggml-cuda/fattn-turbo.cuh`,
`template-instances/fattn-vec-instance-turbo{2,3,4}_0-*.cu` (3, generator-emitted). Edited vendored files
(surgical, default-inert — only fire when `-ctk/-ctv turbo{2,3,4}` is set): `ggml.h`+`ggml.c` (op enum +
`ggml_turbo_wht` ctor), `ggml-cpu/ggml-cpu.c` (abort case + planner n_tasks=1), `ggml-cuda/ggml-cuda.cu`
(TURBO_WHT dispatch + supports_op), `ggml-cuda/fattn-common.cuh` (turbo `get_vec_dot_KQ`/`get_dequantize_V`
cases), `ggml-cuda/fattn-vec.cuh` (`is_turbo_K` config), `ggml-cuda/fattn.cu` (turbo→VEC gate),
`template-instances/generate_cu_files.py` (turbo instance loop), `src/llama-graph.cpp` (`build_attn_mha`
fused gate + prefill-hybrid `TBV_FUSE_MAX_NQ`). f16/q8/q4 paths untouched, boot byte-identical. **Captured
together with Level A in `0073-turboquant-kv` at M6-S3 (#372).** See `poly_kv.md` S2 perf-path Level B.

**F5 M6 PolyKV patch numbering (S3, #372): `0072`/`0073`, NOT the plans' stale `0060`/`0061`.** The
build pipeline applies patches in **pure lexical sort** order (`build-pipeline.ts`, `patches.sort()`);
there is no non-lexical override. The TurboQuant FA-VEC edits (`fattn.cu`, `llama-graph.cpp`,
`ggml-cuda.cu`) sit **on top of** `0070`'s rewrite of those same files, so they MUST sort **after**
`0071` — `0060`/`0061` would sort before `0070` and break the chain (M7 shipped before M6, so the
original plan labels predate it). `0073` also carries the outer-vendor `llamafile/build-functions.sh`
host-build source-list add for the new `ggml-turbo-quant.c` TU. Both patches reverse-apply clean against
live and forward-reproduce the live tree byte-for-byte (snapshot-diff capture, same as `0070`/`0071`).
S0 asymmetric `-ctk q8_0 -ctv q4_0` needs **no patch** (rides M7 dequant-on-lift natively).

**F5 M6-S4 P2 (#406, 2026-06-07): Gemma4 MTP speculative-decode graph + inert (verify-only)
scheduler.** Ported from the AtomicBot `atomic-llama-cpp-turboquant` fork. All new code carries the
`// opencoti F5 M6-S4 mtp` marker (same marker family as P1). **Default-inert:** nothing in the normal
decode/server path invokes the MTP machinery — `llama_decode_mtp` is the only entry and it is called by
nobody in-tree at P2 (active drafter is P3/P4). The patch is **additive-dominant** with **three surgical
edits to existing upstream functions** that must be re-applied on upstream sync (the marker comment is the
real anchor; line numbers below are best-effort):

- **`src/llama-graph.h` — `llm_graph_params::allow_reuse` `token_compat` relaxation** (~line 572). The
  inlined ubatch token/embd compatibility expression is replaced with an MTP-aware named `token_compat`
  bool: MTP graphs feed BOTH token and embd, so steps 1..N-1 of one `decode_mtp` can reuse the encoded
  graph. Default (non-MTP) branch is byte-identical to upstream.
- **`src/llama-graph.cpp` — `llm_graph_result::reset()` nulls `t_argmax`** (~line 805). One added line next
  to the other `t_* = nullptr;` lines so the new MTP-only `t_argmax` field is reset like its siblings.
- **`src/llama-model.cpp` — `build_graph` `case LLM_ARCH_GEMMA4` MTP sub-branch** (~line 8723). The target
  arch case gains a `params.gtype == LLM_GRAPH_TYPE_MTP` branch that builds `llm_build_gemma4_mtp` from the
  nested `mtp_assistant`; the `else` (iswa) path is unchanged. The `LLM_ARCH_GEMMA4_ASSISTANT` case keeps
  its P1 throw-stub.

Everything else is **purely additive** (new functions/fields/file, no upstream-line mutation, marker-tagged):
`LLM_GRAPH_TYPE_MTP` enum value; `llm_graph_input_mtp` class (+`set_input`/`can_reuse`); `t_argmax`
field + `get_argmax()` accessor; `llm_graph_context::build_attn_mtp`; the **new file**
`src/models/gemma4-assistant.cpp` (+ its source-list entry in **both** `llama.cpp/BUILD.mk` —
the authoritative cosmocc build list, the one the host binary actually links — and `src/CMakeLists.txt`
as a mirror; + `llm_build_gemma4_mtp` decl in `src/models/models.h`); `llama_kv_cache::mtp_slot_info` + `llama_kv_cache_iswa::init_mtp`; the synchronous
`llama_context` MTP machinery (`graph_params_mtp`, `ensure_sched_mtp`, `process_ubatch_mtp`,
`graph_compute_mtp`, `decode_mtp` facade → `decode_mtp_sync`) + members `sched_mtp`/`gf_res_prev_mtp`; the
C API `llama_decode_mtp` (+ `include/llama.h` decl); and a `#include "llama-kv-cache-iswa.h"` in
`llama-context.cpp`. **Adaptations vs fork** (documented at the call sites): `build_attn_mtp` passes
`ggml_turbo_wht(..., /*scale=*/nullptr)` instead of the absent `get_turbo_innerq_scale_inv()` (OURS folds
the scale into the device-resident kernel); the `decode_mtp` facade is rerouted unconditionally to
`decode_mtp_sync` (async worker is P4); `ensure_sched_mtp`/`graph_compute_mtp` drop the async-only
`mtp_worker` spawn and `backend_cfg_mu` lock (sync-only P2 path). **Known risk R1 (double-Q-rotation):** if
the MTP cross-read target K/V enter `build_attn_mha` as turbo (fused branch), Q is rotated twice — graph
still *builds* (P2 inert), correctness is a P3 gate.

> **⚠️ RETIRED 2026-06-16 — llamafile 0.10.3 bump (#464, B2).** `0074-speculative-mtp.patch` is **dropped from
> the rebased stack.** The nested llama.cpp base advanced `5e9c63546`→`dbe9c0c` (+613 commits; pin
> `vendors/pin/llamafile.txt` → `7fca8b2  0.10.3`), which ships **upstream-native NextN/MTP**: `255582687`
> "llama + spec: MTP Support (#22673)" (NextN self-attention MTP graph + the `n_rs_seq` recurrent rollback ring,
> a proven ancestor of `dbe9c0c`). The hand-wired `gemma4_assistant` draft head + its `tok_embd`-on-OUTPUT-buft
> placement fix are superseded *for throughput*; **qwen** MTP rebases onto upstream-native `--spec-type
> draft-mtp`, but the **gemma** assistant drafter — which native NextN does **not** cover (Gemma-4 is iSWA,
> no NextN head) — is **restored on 0.10.3 as `0079-gemma-assistant-mtp.patch`**, coexisting as `--spec-type
> draft-assistant` (#423). The 0.10.1-base `0074` *diff* is dropped (it did not apply across the +613-commit
> jump); the *feature* lives on in `0079` — see the active block near the end of this file. The block below is retained for audit history; the patch file no longer
> exists. See `docs/protocols/LLAMAFILE_UPGRADE.md`.

**F5 M6-S4 P3/P4 (#407-410, 2026-06-07): active drafter + throughput win — CAPTURED as `0074-speculative-mtp.patch`.**
P3 activated the synchronous single-token MTP forward (`decode_mtp` → `decode_mtp_sync`,
`process_ubatch_mtp`, the C API) and proved it **logit-equivalent** to plain greedy
(`.opencoti/s4-p3-logit-equiv.sh`: `real_frac=0.0`, 100% draft acceptance). P4 made it a measured
**decode-throughput win**: the assistant's weight-tied dense LM head (`tok_embd`, ~277 MiB) is
classified as an INPUT tensor and was hard-pinned to the CPU buft (`llama-model.cpp:~2716`
`dev_input`), so the per-draft-step full-vocab (262144) output mul_mat ran as a **CPU island** that
stalled the GPU (~95% idle, `GGML_SCHED_DEBUG=2`). The fix (one surgical edit, marker-tagged) routes
the `gemma4_assistant` `tok_embd` through the **OUTPUT** buft (`dev_output`, GPU when the output layer
is offloaded) instead of `dev_input` → MTP-on **114.0 tok/s vs off 78.3 = 1.46×** (seq 113.5;
seq==fused byte-identical; acceptance 0.769/0.606). `0074` reverse-applies clean against live and
forward-reproduces the live tree byte-for-byte (snapshot-diff capture from base `0001-0042 + 0070 +
0071 + 0072 + 0073`, same mechanism as `0070`-`0073`; verified both directions, 24 files, 0 diffs).
TS adapter knobs: `mtpHead`/`specType`/`draftNGpuLayers` (`--mtp-head`/`--spec-type mtp`/`-ngld`) in
`packages/opencoti-llamafile/src/{config,launch}.ts`. The reverted sparse-argmax centroid fast path
(dead for this dense GGUF) is banked at `.opencoti/banked/mtp-sparse-argmax.md`, not shipped.

**F5 M6-S2 #411 (2026-06-07): D=512 fused FA-VEC for turbo2/turbo3 — CAPTURED as `0075-d512-turbo-vec.patch`.**
`0073`'s Level-B fused turbo read covered head_dim∈{128,256}; the Gemma-4 A4B GLOBAL layers use
head_dim 512 (`key_length=512`) and therefore fell back to the Level-A turbo→f16 materialize (a decode
tax that scales with the global-cache `n_kv`). `0075` adds the D=512 fused FA-VEC instances for
turbo2/turbo3 (`DECL_FATTN_VEC_CASE(512, …)` in `fattn-vec.cuh` + the two template-instance `.cu` +
`generate_cu_files.py` autogen line; `fattn.cu` kernel-selection relax + dispatch; `llama-graph.cpp`
`build_attn_mha` `fused_turbo_512` gate). The VEC kernel is already **D-generic** (`nthreads`/
`nthreads_V` derive from D arithmetically; at D=512/ncols=1 the KQ smem is ~4–8 KB and VKQ ~16 regs —
well within the 48 KB static-smem cap, verified compile + run clean). **No new surgical hook** — every
edit sits inside `0073`'s existing turbo regions; **no new ABI; no new flag** (the D=512 fused path
auto-engages for turbo2/3 decode via the existing `TBV_FUSE_MAX_NQ` knob, default 8). Correctness is
**Parseval-EXACT** (the WHT is block-diagonal over 128-blocks, so D=512 = 4 Parseval-exact 128-blocks):
fused-vs-materialize logit-equivalence `real_frac=0.0` for BOTH turbo3 and turbo2 (`.opencoti/p411-d512-gate.sh`).
Perf — turbo3 @ ctx 27.7k, fused **41.0 vs materialize 29.7 tok/s = +38%**, scaling with `n_kv`
(`.opencoti/p411-perf.sh`). Snapshot-diff capture (6 files, 95 lines, 0 new) from base
`0001-0042 + 0070 + 0071 + 0072 + 0073 + 0074`; verified both directions, 0 diffs. **No TS adapter
change** (no user-facing flag).

When you add a hook, append a row above, replacing the placeholder
with: stable hook id, file path + best-effort line number (line moves
on upstream syncs are normal — the marker comment is the real
anchor).

## Why we're strict

Every undiscoverable edit to an upstream file is a future merge
conflict that someone (probably us) will resolve at 2 AM during a
release crunch. Strictness up front is the cheapest tax.

### `opencoti F5 gemma4-shared-kv` (F5 — elastic E-series KV-layer-sharing) — ⚠️ RETIRED 2026-06-16

> **RETIRED at the llamafile 0.10.3 bump (#464, B2).** `dbe9c0c` loads gemma4
> `gemma4.attention.shared_kv_layers` natively (its loader no longer requires `attn_k`/`attn_k_norm` on every
> layer), so the `TENSOR_NOT_REQUIRED` loader gate described below is **superseded by upstream native** support
> and was dropped from the rebased stack. `0076-gemma4-shared-kv.patch` no longer exists; entry kept for audit
> history. See `docs/protocols/LLAMAFILE_UPGRADE.md`.

One *vendored-source* marker (`// opencoti F5 gemma4-shared-kv (#417)`) in
`vendors/sources/llamafile/llama.cpp/src/llama-model.cpp`,
in the `LLM_ARCH_GEMMA4` `create_tensor` loop. The elastic Gemma-4 E-series (E2B/E4B) declare
`gemma4.attention.shared_kv_layers` (E2B 20/35, E4B 18/42): the trailing layers carry no
`attn_k`/`attn_k_norm` and reuse an earlier layer's KV cache. The graph side
(`src/models/gemma4-iswa.cpp` `hparams.has_kv(il)` guard), `hparams.has_kv()`
(`src/llama-hparams.cpp`), and the KV-cache layer-reuse callback (`llama-model.cpp`, the
`arch == LLM_ARCH_GEMMA4` `layer_reuse_cb`) were already brought over by the MTP/TurboQuant
port; the only missing piece was the loader requiring `attn_k`/`attn_k_norm` on every layer.
The hook gates both with `const int kv_flags = hparams.has_kv(i) ? 0 : TENSOR_NOT_REQUIRED;`.
Inert for dense Gemma-4 (A4B/12B/31B): they set `n_layer_kv_from_start = n_layer`, so
`has_kv(i)` is true for all `i` and `kv_flags ≡ 0` — the `create_tensor` calls are bit-identical
to upstream. **Captured as `0076-gemma4-shared-kv.patch`** (snapshot-diff vs the 0075-applied
base; reverse-applies clean + forward-reproduces live byte-for-byte). Unblocks E2B/E4B load
(was bug-438). Marker convention: this is a **vendored-source** `// opencoti <feature>` marker
(patches/README §"In-tree markers vs surgical hooks") — NOT an `// opencoti-hook:` marker, so the
`git grep "opencoti-hook:" packages/ docs/` inventory is unchanged.

### `opencoti F5 M6-S4 mtp ordered-gpu` (F5 M6-S4 — ordered-embeddings draft-head GPU placement) — ⚠️ RETIRED 2026-06-16

> **RETIRED at the llamafile 0.10.3 bump (#464, B2).** This gemma4-MTP ordered-drafter GPU-placement hook is a
> follow-on to `0074` and is superseded by the same **upstream-native NextN/MTP** convergence (`255582687`,
> #22673) under `dbe9c0c`. `0077-gemma4-mtp-ordered-gpu.patch` was dropped from the rebased stack; entry kept
> for audit history. See `docs/protocols/LLAMAFILE_UPGRADE.md`.

One *vendored-source* marker (`// opencoti F5 M6-S4 mtp ordered-gpu (#418)`) in
`vendors/sources/llamafile/llama.cpp/src/llama-model.cpp`, in the
`LLM_ARCH_GEMMA4_ASSISTANT` `create_tensor` path inside `if (hparams.use_ordered_embeddings)`.
The follow-on to 0074's dense `tok_embd` fix: the ordered ("efficient") draft head reads
`mtp_centroids` (centroid mul_mat) and `mtp_token_ordering` (get_rows) every draft step, and
llama.cpp default-classifies both as inputs pinned to the CPU buft (`dev_input`) — a CPU island
that fragments the MTP graph (`GGML_SCHED_DEBUG` splits=2, assistant `CPU_Mapped` 277 MiB) and
makes MTP-on *slower* than off on the ordered drafters (E2B/E4B). The hook routes both through the
OUTPUT buft (`pimpl->dev_output.buft_list`, GPU when the output layer is offloaded) via the
`ml.create_tensor(hparams, &pimpl->cpu_buft_list, dev_output…)` form — identical mechanism to 0074.
Result: assistant `CPU_Mapped` 277→0.80 MiB, draft `dur(g)` 783→300 ms; E2B 0.87×→1.22×,
E4B 1.34×→1.42× (RULER 32k, q8_0 KV; acceptance bit-identical pre/post). **Inert for the dense
drafters** (A4B/12B/31B): they set `use_ordered_embeddings=false` and never enter this branch.
**Captured as `0077-gemma4-mtp-ordered-gpu.patch`** (snapshot-diff vs the 0076-applied base;
reverse+forward byte-identical). Vendored-source marker — not counted by the `opencoti-hook:`
inventory.

### `opencoti F5 dca` (F5 — Dual Chunk Attention, training-free long-context)

The DCA marker family `// opencoti F5 dca` (187 occurrences) spans the vendored llama.cpp tree —
this is a substantial additive feature, not a one-line hook. Surface:

- **New files** `vendors/sources/llamafile/llama.cpp/src/dca.{cpp,h}` — the three-regime builder
  `build_attn_dca` / `build_attn_dca_core` / `build_attn_inp_dca` and the
  `ggml_flash_attn_ext_dca_fused` host wrapper (INTRA `pos%c` / SUCC `+c` / INTER `2c`, merged by
  online-softmax).
- **KV-cache fill** `src/llama-kv-cache.cpp` `set_input_dca` — #635 INTER-Q = `2c` dense
  position-based fill + the #617B sticky `is_fragmented` fragmented-mask fallback (masks left NULL on
  the contiguous fast path).
- **Per-arch dispatch hooks** in `src/models/{gemma4-iswa,qwen2,qwen3,qwen35,qwen35moe,qwen3moe}.cpp`
  (+ `models.h`, `llama-arch.cpp`, `llama-graph.{h,cpp}`, `llama-kv-cache.h`, `llama-kv-cells.h`,
  `llama-context.cpp`) — gemma4 hd512 (global layers), qwen2/3 hd128, qwen3.5 family hd256, incl. the
  qwen35 `nextn_predict_layers` MTP-block skip (#622).
- **Runtime knobs** `src/llama-cparams.h` (`dca_enabled` / `dca_chunk_size` / `dca_yarn_factor`) + `common/{arg,common}.{h,cpp}`
  args `--dca` / `--dca-chunk-size` / `--dca-yarn-factor` (env `LLAMA_ARG_DCA*`); `tools/server/server-context.cpp`.
  `--dca-yarn-factor` is **consumed** in `build_dca_rope` (`dca.cpp`) as the YaRN mscale —
  `dca_attn_factor = attn_factor * cparams.dca_yarn_factor` into `ggml_rope_multi`/`ggml_rope_ext`
  (marker `// opencoti-hook: dca-yarn-factor (bug-740 / #550)`, patch `0087`; default 1.0 ⇒ inert/byte-identical).
  **Characterized as no-win for retrieval** (over-sharpens; default 1.0 correct — see README `0087` row).
- **New ggml op** `GGML_OP_FLASH_ATTN_EXT_LSE` (`ggml/include/ggml.h` + `ggml.c`, `ggml-backend.cpp`,
  `ggml-cpu/*`, and the fused MMA kernel + LSE combine in `ggml/src/ggml-cuda/{fattn*,ggml-cuda.cu}`):
  flash-attention emitting a NORMALIZED O packed with per-row lse, so the three regimes merge exactly.

Default-OFF (`cparams.dca_enabled=false`) is byte-identical to pre-0078 (the contiguous fast path leaves
the fragmented masks NULL). Constraints today: `head_dim` in {128,256,512}, `DK==DV`, **f16 KV required**
(`dca.cpp` assert — to be generalized to all KV types, incl. bf16 / scalar / turbo / TCQ, in the follow-on
all-KV-types milestone). **Captured as `0078-dca.patch`** (clean-room capture from the sibling
`opencoti-dca` repo, 37 files / 2 new; landed byte-identical, `git apply --check` clean on the
0077-applied tree, production-clean of `DCA617*` debug). Marker convention: these are **vendored-source**
`// opencoti <feature>` markers (patches/README §"In-tree markers vs surgical hooks") — there are **zero**
`// opencoti-hook:` markers in 0078, so the `git grep "opencoti-hook:" packages/ docs/` inventory is
unchanged. **ABI:** adds the `GGML_OP_FLASH_ATTN_EXT_LSE` ggml op — the cosmocc binary + `ggml-cuda.so`
DSO must be rebuilt and paired.

### `opencoti F5 M6-S4 mtp (0.10.3 restore)` (F5 M6-S4 — Gemma-4 assistant-drafter MTP, restored onto 0.10.3) — `0079`

The gemma assistant drafter retired as `0074` at the 0.10.3 bump (its 0.10.1-base snapshot-diff did not apply
across the +613-commit jump to `dbe9c0c`) is **restored on the new base as `0079-gemma-assistant-mtp.patch`**
(#423), re-exposing it as a first-class `--spec-type draft-assistant` that **coexists** with upstream-native
NextN MTP (`--spec-type draft-mtp`). This is a deliberate **two-engine union**, not a reversal of the
retirement: native NextN covers the qwen-family NextN archs; the assistant drafter covers **Gemma-4**, which
has **no native NextN head** (iSWA) — so for Gemma it is the *only* MTP path. The user mandate (#423) is
explicit: do not drop ours; extend/restore it so it works alongside every other development (PolyKV,
rolling-KV, turbo, DCA).

**No enum/graph collision:** `COMMON_SPECULATIVE_TYPE_MTP` (ours) vs `COMMON_SPECULATIVE_TYPE_DRAFT_MTP`
(native); `LLM_GRAPH_TYPE_MTP` (ours) vs `LLM_GRAPH_TYPE_DECODER_MTP` (native). The ~0.4B Q-only
cross-attention drafter (`LLM_ARCH_GEMMA4_ASSISTANT`) loads **into** the gemma4 target via
`llama_model_load_mtp_from_file` (no second `llama_context`, no second KV cache) and reads the target KV
read-only at decode (`build_attn_mtp`). It rides the 0.10.3 **seq-aware speculative framework**
(`common_speculative_process`/`_draft`/`_accept`/`_need_embd` public hooks, which replace the old server
`h_idx`/`seq_id`/`set_h_idx` plumbing) — which is *why* it composes with PolyKV / multi-seq for free.

All new code carries the `// opencoti F5 M6-S4 mtp` marker (vendored-source, **not** `opencoti-hook:` —
registered via the `0079` README row + this block; the `packages/ docs/` counter is unchanged). The patch is
**additive-dominant**; the surgical edits to existing 0.10.3 upstream functions that must be re-applied on the
next upstream sync (the marker comment is the real anchor; line numbers best-effort):

- **`src/llama-graph.cpp` — `llm_graph_input_attn_kv_iswa::set_input` buffer-guard decouple** (**bug-508**).
  0.10.3 nests the `set_input_kq_mask` call **inside** `if (self_k_idxs && self_k_idxs->buffer)`. The
  read-only MTP draft graph (`build_attn_mtp`) prunes the KV-write idxs (their galloc buffer is null) yet still
  reads the mask — so the nested guard would skip the *live* mask and feed garbage (a silent re-introduction of
  bug-404). Fix: **decouple** — each `self_{k,v}_idxs` / `self_kq_mask{,_swa}` / rot setter now guards on **its
  own** `->buffer`. **Byte-identical for every non-MTP graph** (all buffers live ⇒ every branch taken, as
  upstream).
- **`tools/server/server-context.cpp` — leading `has_dft() && draft-assistant` branch.** Before the existing
  `has_dft()` draft-model path (now `else if`), a new branch detects `COMMON_SPECULATIVE_TYPE_MTP` in
  `--spec-type`, calls `llama_model_load_mtp_from_file(model_tgt, params_dft.model.path, mparams_dft)`, and
  leaves `model_dft`/`ctx_dft` **null** (null-safe: every `ctx_dft` checkpoint path early-returns on a null
  context).
- **per-arch loader + graph-dispatch plumbing** — a per-model class `llama_model_gemma4_assistant` registered
  via `llama_model_mapping` (the 0.10.1 giant-switch loader is gone) and a `params.gtype == LLM_GRAPH_TYPE_MTP`
  sub-branch routing to `llm_build_gemma4_mtp` (new file `src/models/gemma4-assistant.cpp`); the base gemma4
  iSWA path is unchanged.

Everything else is additive (the `LLM_GRAPH_TYPE_MTP` enum, `llm_graph_input_mtp` class +
`set_input`/`can_reuse`, `build_attn_mtp`, the `mtp_slot_info`/`init_mtp` KV machinery, the C API
`llama_decode_mtp`, and the `--spec-type draft-assistant` arg). **Default-inert:** the MTP path is reached only
with the assistant loaded + `--spec-type draft-assistant`; plain decode and native `draft-mtp` are unchanged.

**Touches zero `.cu`/`.cuh`** — the assistant rides the existing fattn/turbo/DCA CUDA kernels, so landing it on
the live tree is a **cosmocc host rebuild only** (no DSO rebuild/restamp). Capture: snapshot-diff from the spike
`0078`-tip base (23 files, 1 new `src/models/gemma4-assistant.cpp`, 0 index lines); **reverse + forward
byte-identical** in the spike; spike **host build GREEN** (compiles + links; `strings` shows `draft-mtp` and
`draft-assistant` coexisting). **Live-binary composition gates are PENDING** (#461): assistant-MTP × {PolyKV /
POSITION_WINDOW / turbo / DCA} logit-equivalence (`real_frac=0`, the locked "non-MTP works in MTP" gate) runs on
the live CUDA binary, gated on a host rebuild + GPU availability. **Applies after `0078`.**

#### Upstream-alignment overlay (#476 — adopt upstream `#23398`/`#24282` DATA-plane, keep our control-plane)

Upstream llama.cpp merged Gemma-4 MTP (**PR #23398** = release **b9549**, + E-series **#24282**). `0079` now
carries upstream's **data-plane identity** so a future llamafile bump onto a base containing #23398 reconciles
the gemma assistant-MTP as a clean *delete of the identity layer*, keeping only our control-plane delta. The
rename folded into `0079` (S1; re-runs of the same hunks, no new files):

| surface | was (ours) | now (upstream-aligned) |
|---|---|---|
| arch string | `gemma4_assistant` | **`gemma4-assistant`** (hyphen) |
| proj tensors | `mtp.pre_projection` / `mtp.post_projection` | **`nextn.pre_projection`** / **`nextn.post_projection`** (`LLM_TENSOR_NEXTN_PROJ_{PRE,POST}`) |
| E-series tensors | `mtp.centroids` / `mtp.token_ordering.weight` | **`masked_embd_centroids`** / **`masked_embd_ordering`** (bare, NO `.weight`) (`LLM_TENSOR_MASKED_EMBD_*`) |
| backbone-width KV | `gemma4_assistant.n_embd_backbone` | **`%s.embedding_length_out`** (`LLM_KV_EMBEDDING_LENGTH_OUT`); hparams `n_embd_out_impl` |
| layers KV | (derived) | adds **`%s.nextn_predict_layers`** (= block_count) |

**What we deliberately do NOT adopt — `ctx_other`.** Upstream's assistant mechanism is a **second `llama_context`**
that aliases the target KV by struct-copying its layer record + mirroring cell metadata each step
(`llama-kv-cache.cpp` `apply_ubatch: if (other) { v_cells = other->v_cells; return; }`; `seq_*`/`update` all
`if (other) return;`). It assumes the target KV is **vanilla: single-owner, contiguous, unquantized** — the
opposite of opencoti's KV moat, and it is *independently broken* under quantized KV (public PR: 0% acceptance
with `-ctk q8_0 -ctv q8_0`). It would conflict with **all five** gate-required features (PolyKV per-stream
layers, rolling-KV POSITION_WINDOW, M2/M3 head-split, TurboQuant KV, DCA). So we **keep the opencoti
control-plane**: single-context `build_attn_mtp` read-only cross-read (`ctx_dft == nullptr`, load INTO target via
`llama_model_load_mtp_from_file`); ordered/centroid masked-embedder head; `tok_embd`→OUTPUT buft; fused N-step;
`--spec-type draft-assistant` (CLI string, not GGUF identity — preserves the 2-engine dispatch vs qwen
`draft-mtp`). Bespoke keys with no upstream equivalent (`n_centroids`, `centroid_top_k`, `use_ordered_embeddings`,
`attention.k_eq_v`, `requires_target_arch`) remain opencoti-private under the `gemma4-assistant.*` prefix —
upstream loaders ignore unknown keys.

**The rename is semantically INERT** — proven at three levels: (1) the re-formatted A4B drafter GGUF
(`.opencoti/gguf-upstream-rename.py`, Q8_0 bytes copied verbatim) has whole-file tensor-data sha256 == source;
(2) live boot-smoke on the renamed CUDA host binary loads `gemma4-assistant`, routes `draft-assistant`, drafts
(44% accept), decodes coherently; (3) **`real_frac = 0` logit-equivalence** on the renamed engine
(`.opencoti/s4-p3-logit-equiv.sh`, A4B v6-coder Q4_K_M). The `0079` patch is **surgically confined**: 16 of its
23 file-stanzas are **byte-identical** to the pre-rename `0079`; only the 7-file data-plane surface
(`llama-arch.{cpp,h}`, `llama-context.cpp`, `llama-hparams.h`, `llama.cpp`, `common/speculative.cpp`,
`src/models/gemma4-assistant.cpp`) changed — the entire control-plane (kv-cache, graph, model registration,
server branch, base gemma4) is untouched. **Chain proof (G33/G35):** `0079` regenerated as
`diff(clean-0078-replay, live)` where the replay is a fresh `git archive HEAD` + llamafile `apply-patches.sh`
overlay + strict `git apply` of `0001..0078`; the FULL chain `0001..0079` then strict-applies from pristine
(**0 rejects**), final tree **== live byte-identical across 2941 source files**, reverse round-trips. **A4B
beachhead only** — re-formatting the other 4 drafters (E2B/E4B/12B/31B) + the full composition matrix on the
renamed engine are FOLLOW-ON (#461/#476). Touches zero `.cu`/`.cuh` (host rebuild only).

**`#408` OUTPUT-buft placement re-claimed on 0.10.3 (#484).** The `tok_embd`→OUTPUT-buft win listed in the kept
control-plane is not free on the bumped base: 0.10.3's `ml.create_tensor` selects the device buft by the tensor's
`llm_tensor_info` layer, and `TOKEN_EMBD` classifies `LLM_TENSOR_LAYER_INPUT` → **CPU** buft, so the per-model
loader fragments the assistant's per-step full-vocab matmul into a CPU island (MTP-on *slower* than off at short
ctx; bug-540). Re-claimed with an **additive** `llama_model_base::create_tensor_output_head(...)` (+ml-less overload,
`src/llama-model.{cpp,h}`) that passes `dev_output.buft_list` in the INPUT slot to force GPU OUTPUT placement, with
**normal** load accounting (NOT `TENSOR_DUPLICATED` — `tok_embd` is the sole-use tied output head). Gated:
`real_frac = 0` (placement is byte-equivalent) **+ 1.57× at RULER-32k** (off 56.6 → on 88.8 tok/s, 87% accept) —
exceeds the historical 1.46×. Folded into `0079` (still 23 files; the 3 stanzas `llama-model.{cpp,h}` +
`gemma4-assistant.cpp` grew, full `0001..0079` chain re-proven from pristine).

**PolyKV multi-slot support folded into `0079` (#485).** Two surgical hooks in
`tools/server/server-context.cpp` (both `// opencoti F5 M6-S4 mtp (#485)`) let the Gemma assistant
drafter compose with the SharedKVPool multi-tenant path: (1) at the spec-fit pre-measure (~:809),
`spec_assistant_fit` skips the standalone draft-`llama_context` memory probe for the assistant engine
(`gemma4-assistant` legitimately refuses to be a primary `-m` model, so the probe would log a benign
`failed to measure draft model memory` warning — the source of the earlier bug-550 misdiagnosis); (2) at
the assistant-load branch (~:940), a boot guard requires `--kv-unified` when `--parallel >1`, because the
single-context cross-read (`build_attn_mtp` → `build_attn_mha`) infers the stream count from the target
K's 4th axis (`n_stream = k->ne[3]`, `src/llama-graph.cpp:2115`) and the per-stream-split KV layout
(`0031`) hands back a multi-stream tensor the single-token draft query cannot be partitioned against
(`ggml.c:3754` reshape assert). PolyKV forces `--kv-unified`, so it composes (`real_frac=0` + ~93% draft
acceptance @ `--parallel 2 --kv-unified`); the guard turns the unsupported non-unified multi-slot config
into a clean boot error instead of a mid-request SIGABRT. Within the existing `0079` 23-file surface (the
two hooks live in the already-touched server stanza); full `0001..0079` chain re-proven byte-identical
from pristine. Non-unified multi-slot (an `n_seq`-aware assistant draft) is tracked under #469.

### `0080-mtp-d512-fa` — D512/gqa≤2 flash-attn for the Gemma-4 assistant-MTP drafter (#491+#493, bug-567)

The Gemma-4 E4B assistant-MTP drafter's single GLOBAL layer is head_count 4 / head_count_kv 2 →
**gqa_ratio 2 at head_dim 512**, a kernel-coverage hole on the 0.10.3 base: neither the MMA
(`switch_ncols2`, `fattn.cu`) nor the TILE (`switch_ncols2`, `fattn-tile.cuh`) D512 path instantiates
`ncols2 ∈ {1,2}` (gated `DV≤256` for binary size), so a D512/gqa≤2 `FLASH_ATTN_EXT` op aborts at runtime
(**bug-567**). Every other Gemma-4 config is gqa≥4 (A4B/12B/31B + E4B base; A4B globals are head16/kv2 →
gqa 8) and unaffected.

`#491` was a host stop-gap: a `build_attn_mha` guard routed the D512/gqa≤2 layer to the existing non-FA
explicit-softmax branch (correct, but "fa off" on the hot path). `#493` is the real fix — a genuine
D512 f16 flash-attn via the VEC kernel — and **retires** the `#491` non-FA route for the f16 case.

**Vendored-source markers `// opencoti #491` / `// opencoti #493`** (5 files, all nested; **not** an
`opencoti-hook:` — additive vendored-source edits, registered here):

- `ggml/src/ggml-cuda/template-instances/fattn-vec-instance-f16-f16.cu` + `…/generate_cu_files.py` —
  `DECL_FATTN_VEC_CASE(512, GGML_TYPE_F16, GGML_TYPE_F16)` instance + the autogen-emitter line.
- `ggml/src/ggml-cuda/fattn-vec.cuh` — `extern DECL_FATTN_VEC_CASE(512, F16, F16)`.
- `ggml/src/ggml-cuda/fattn.cu` — **two** head-size switches both get the D512 f16 case: the selector
  (`ggml_cuda_get_best_fattn_kernel`: `Q->ne[0]==512 && gqa_ratio≤2 && gqa_opt_applies && K/V f16 →
  BEST_FATTN_KERNEL_VEC`) **and** the VEC dispatch (`ggml_cuda_flash_attn_ext_vec`, `FATTN_VEC_CASE(512,
  F16, F16)` before the `GGML_ABORT`). Patching only the selector makes it pick VEC while the dispatch
  falls through to the abort → a **decode-time** `fattn.cu` fatal (boot is clean; it crashes on the first
  global-layer decode). Both are required.
- `src/llama-graph.cpp` — narrow the `#491` guard: `fa_no_gpu_kernel` now requires K/V **not** f16/f32
  (`fa_d512_f16_vec`), so f16 D512/gqa≤2 flash-attns on-GPU and only quantized/bf16 D512/gqa≤2 (which the
  assistant drafter never produces) keeps the soft_max fallback.

The VEC kernel tiles `Q->ne[1]` (`cols_per_block` 1 decode / 2 prefill+verify), so the single f16 D512
instance is a real flash-attn at **any n_q** (decode + prefill + the spec-verify batch). `logit_softcap`
is forced off for `D∉{128,256}` in the VEC kernel — safe because Gemma-4 `attn_soft_cap` is false.

**Gate** (`.opencoti/m493-fa-d512-gate.sh`): E4B `-fa on` assistant-MTP decodes with **no abort** (147.86
tok/s); the verified-greedy output is **byte-identical** to the `#491` soft_max path and draft acceptance
is unchanged (**0.67910** both) — assistant-MTP is exact spec-decode, so a wrong FA cannot corrupt the
verified output but *would* collapse acceptance; equal acceptance + byte-equal prefix ⇒ the VEC FA is
numerically correct. A4B (gqa 8, untouched MMA) regresses nowhere. Both DSO paths rebuilt byte-identical
(sha256 `a8ddcd82`). **Chain proof:** snapshot-diff capture of 5 files (0 new) from the post-`0079` base
(vendor backup `20260618-054922`, a real `#491=0/#493=0` post-0079 tree); `git apply --check` clean +
forward-reproduces live byte-for-byte. Touches `.cu`/`.cuh` ⇒ CUDA DSO rebuild + dual restamp required to
land live. **Applies after `0079`.** Provenance: opencoti-original.

### `0081-tcq-q6-asym-kv` — TCQ KV tiers + q6_0 scalar KV + asymmetric TCQ FA-VEC (#447/#511/#512/#513/#515)

Three additive KV-quant features landed as one snapshot-diff patch (they share the FA-VEC instance surface
and a single host enum bump, so they cannot be split without an intermediate `GGML_TYPE_COUNT` skew):

1. **TCQ tiers** — `GGML_TYPE_TURBO3_TCQ = 48` (3.25 bpv; 512-state Viterbi, k=3/L=9) and
   `GGML_TYPE_TURBO2_TCQ = 49` (2.25 bpv; 256-state, k=2/L=8). Trellis-Coded Quantization: a GPU FWHT-whitened
   **Viterbi-trellis encode** (`turbo-tcq-cuda.cuh` `k_set_rows_turbo{3,2}_tcq`, dispatched from `set-rows.cu`)
   + an **O(1) in-register codebook decode** fused into FA-VEC (`fattn-tcq.cuh` readers, smem codebook headers
   `tcq-decode-cb.cuh`). Codebooks are real-27B-trained (rebuild-free reloadable). 27B KLD **0.0366** at
   3.25 bpv beats plain turbo3_0 **0.0470** at iso-bits — TCQ earns its keep on the head_dim-128 target.
2. **q6_0** — `GGML_TYPE_Q6_0 = 50` (6.5 bpv), a scalar KV type ported from `Anbeeld/beellama.cpp` (⊃
   `/shared/dev/ik_llama.cpp`, `block_q6_0 {ggml_half d; uint8_t qh[8]; uint8_t qs[16];}` QK6_0=32). 27B KLD
   **0.01973** == q5_0 — it sits on the K-precision-dominant frontier (asymmetric strong-K combos like
   `q8_0/q6_0` beat symmetric at equal size). `GGML_TYPE_COUNT` bumps 48 → **51**.
3. **Asymmetric TCQ** — the mixed `turbo3_tcq↔turbo2_tcq` FA-VEC instances (both directions), strong-K/cheap-V.

**9 keeper FA-VEC instances** (the Anbeeld set, not the full cross-product, to bound compile blowup):
q6_0 ×5 (`q6_0/q6_0`, `q8_0/q6_0`, `q6_0/q5_0`, `q6_0/f16`, `f16/q6_0`) + TCQ ×4 (`turbo3_tcq`/`turbo2_tcq`
symmetric + both asymmetric). All read through **GPU FA-VEC** — `GGML_CUDA_FA_ALL_QUANTS` is the shipped
default (#531, `build-pipeline.ts --fa-all-quants`), and for the non-all-quants build the 9 instances are
force-compiled by `llamafile/build-functions.sh collect_gpu_sources` blocks 3b/3c/3d (the **#468/bug-339**
dlopen-symbol guard: without them the DSO lacks `ggml_cuda_flash_attn_ext_vec_case<D,q6_0|tcq,*>` and dlopen
fails the instant such KV reaches flash-attention; each is `[ -f ]`-guarded → no-op on a vanilla tree). A
`TURBO_TCQ_DUMP_ACTS=<path>` env hook (#515) dumps post-FWHT whitened activations for rebuild-free codebook
retraining (zero overhead unset).

The earlier `turbo3_tcq_w` 1024-state free-widen tier (#514) was measured **inert** and **removed** (#527) —
it is **not** in this patch (no `_w` enum/instances/kernels).

**Vendored-source markers `// opencoti #447` / `#511` / `#512` / `#513` / `#515`** (38 files, all under
`vendors/sources/llamafile/` — **not** `opencoti-hook:` markers; these are additive vendored-source edits,
registered here, and do **not** change the `git grep "opencoti-hook:" packages/ docs/` count):

- **9 NEW** `ggml/src/ggml-cuda/template-instances/fattn-vec-instance-{q6_0-q6_0,q8_0-q6_0,q6_0-q5_0,q6_0-f16,
  f16-q6_0,turbo3_tcq-turbo3_tcq,turbo2_tcq-turbo2_tcq,turbo3_tcq-turbo2_tcq,turbo2_tcq-turbo3_tcq}.cu`.
- **3 NEW** `ggml/src/ggml-cuda/{fattn-tcq.cuh, tcq-decode-cb.cuh, turbo-tcq-cuda.cuh}`.
- Host enum/traits/quants: `ggml/include/ggml.h`, `ggml/src/{ggml.c, ggml-common.h, ggml-quants.c,
  ggml-quants.h, ggml-turbo-quant.c}`, `ggml/src/ggml-cpu/{ggml-cpu.c, quants.h}`, `common/arg.cpp`
  (`kv_cache_types += {Q6_0, TURBO3_TCQ, TURBO2_TCQ}`), `tools/perplexity/main.cpp`.
- CUDA glue: `ggml/src/ggml-cuda/{common.cuh, convert.cu, cpy.cu, cpy-utils.cuh, dequantize.cuh, fattn.cu
  (FATTN_VEC_CASES + supports_op), fattn-common.cuh, fattn-vec.cuh, ggml-cuda.cu (SET_ROWS supports_op),
  set-rows.cu}`, `ggml/src/iqk/{iqk_common_extra.h, iqk_ggml_type_ext.h}`.
- Graph/DCA lift: `src/{llama-graph.cpp, dca.cpp}` (scalar q6_0 + asym compose with DCA via the #444
  f16-dequant-on-lift; **TCQ aborts under DCA**, rc=134, pending #505).
- Build glue: `BUILD.mk`, **outer** `llamafile/build-functions.sh`.

**Correctness** via KLD / logit-equiv / RULER niah — never greedy needle (bug-263/270). RULER niah 8 tiers
**100/100** @ 4k–512k (#529 §5b). PPL/KLD valid **only** on Qwen3.6-27B-Omnimerge (head_dim 128); A4B raw-PPL
is garbage (bug-631). TCQ pays an in-register-Viterbi decode tax (~15% slower tg on weight-bound 27B, ~45% on
small A4B). **Chain proof:** snapshot-diff capture (38 files, 9 NEW) from the post-`0080` BASE (vendor backup
`20260622-111006-pre-tcq-q6-asym-capture`); applying `0081` to a clean `apply(0001..0080)` tree
**byte-identically reproduces GOAL** (full-tree `diff -rq` empty, nested + outer). Touches `.cu`/`.cuh` ⇒ CUDA
DSO rebuild + dual restamp required to land live. **Applies after `0080`.** Provenance: opencoti-original
(q6_0 ported from Anbeeld/beellama.cpp ⊃ ik_llama.cpp).

### `0089`/`0093`–`0095` — bug-858 MTP parity ship + bug-2108 chain repair (2026-07-05, #608/#614)

**bug-2108 chain repair (renumbering).** The committed chain no longer
strict-applied from clean 0.10.3: `0089-rolling-kv-compute-reserve` (and later
patches) were snapshot-captured from a tree carrying **uncaptured** work
(sparse-attn vslash #551/#557–568 incl. the `sparse_attn_block_size` KV-ctor
plumbing, #591/#592 decode-overhead work, bug-1840 fixes), so their context
lines baked in content no patch introduced. Repair:
`0089-sparse-attn-vslash-foundation` (NEW — `diff(chain-0088 baseline,
vendor-backup 20260701-123044)`, proven to be exactly old-0089's capture base);
old `0089`→`0090`, `0090`→`0091`; old `0091-rolling-kv-diag-visibility`
**retired** (its `llama-kv-cache.cpp` hunks folded into `0093`). Full-chain
proof: clean `git archive` 0.10.3 + Mozilla `llama.cpp.patches` overlay +
strict `git apply` of all 41 patches ⇒ **byte-identical** to the live vendored
tree (excl. `tools/ui/dist` fetch-time assets + submodules). Lesson (bug-250
class): run the chain proof at EVERY capture, not once per era.

**Markers registered by this ship:**

- `// opencoti-hook: sparse-attn-vslash` (`common.h` + graph/KV plumbing) —
  vertical-slash sparse attention foundation; patch `0089`.
- bug-858 dual-ctx comment blocks (`// opencoti bug-858 dual-context MTP …`)
  across `speculative.cpp`, `llama-context.*`, `llama-kv-cache*`,
  `llama-model.cpp`, `models/gemma4-assistant.cpp`, `server-context.cpp` — the
  `ctx_other`/`mem_other`+`share` drafter-shares-target-KV port; patch `0093`.
  Control-plane stays opencoti (`--spec-type draft-assistant --mtp-head` +
  `OPENCOTI_MTP_DUAL_CTX=1`); data plane aligned with upstream #23398/#24282.
- `// opencoti-hook: mtp-verify-mmvf (bug-2103)` (`ggml-cuda/mmvf.cu`) —
  TinyBLAS-gated `ne11≤8` mmvf cap for the spec-verify GEMV cliff; patch `0093`.
- `// opencoti-hook: tinyblas-small-gemm (bug-2104)` (`llamafile/tinyblas.cu`)
  — second small-GEMM config when the 128×64 config yields <24 blocks; patch
  `0094`. NOTE: outer-llamafile file (not llama.cpp) — survives llama.cpp
  bumps, conflicts only on llamafile-proper bumps.
- `// opencoti-hook: ckpt-buf-noinit (#614/bug-2107)` (`common.h`,
  `server-task.h`, `server-context.cpp`) — `common_state_buf` + checkpoint
  buffer recycle pool; rides with `checkpoint_min_step` 256→8192 (bug-2105,
  plain default alignment, no marker — registered via this row); patch `0095`.
  Future-bump gotcha (bug-2106): the struct default is header-inlined into
  EVERY constructing TU (incl. `o/llamafile/*.o`) — after editing it, purge all
  constructing objects (both arches, `CCACHE_DISABLE=1`) and gate on the boot
  line `min spacing = 8192`, never on "build succeeded".

### `0096` — single-context Gemma MTP facade REMOVED (#611, 2026-07-05)

END of the bug-858 port. The dual-context assistant (patch `0093`) is now the
ONLY `--spec-type draft-assistant` execution path — the `OPENCOTI_MTP_DUAL_CTX`
env gate and the whole single-context in-target engine are deleted (~1600
lines: `decode_mtp*` facade/sync/fused + async worker, `process_ubatch_mtp`,
`sched_mtp`, `llama_decode_mtp` C API, iswa `init_mtp`, `build_attn_mtp`,
`llm_build_gemma4_mtp`, `common_speculative_impl_draft_assistant`).

**Registry deltas:** the `opencoti F5 M6-S4 mtp` markers covering those
functions are gone with them; the surviving assistant surface is the loader
(`llama_model_load_mtp_from_file` / `mtp_assistant`), the dual-ctx graph in
`models/gemma4-assistant.cpp`, and the `0093` dual-ctx comment blocks. The
NextN fused-draft markers (`fused-nextn-mtp`, `mtp-verify-mmvf`) are untouched.
A future llamafile bump reconciles the assistant almost entirely against
upstream #23398/#24282 (data plane AND execution shape now upstream-aligned;
remaining opencoti-specific: control-plane flags + single-binary loading).

Gate: A4B n2 305.4 tps / 0.849 accept == 0093 reference (boot line
`dual-context draft created` without any env); q35 NextN n3 248.8 / 0.662.
bug-2110 (apparent accept 0.635 at `--parallel 2 --kv-unified`) was RESOLVED
same-day as a tps-harness artifact (degenerate-filler prompt): on the real
9-prompt mtp-bench the dual-ctx assistant holds accept at par2 (ours
0.785/0.783 par1/par2; upstream 0.793/0.792) — PolyKV × assistant-MTP composes
cleanly on the dual-ctx path.

## 2026-07-05 — 0097 native-ELF CMake lane (#619, secondary artifact of #613)

`0097-native-elf-cmake.patch` makes the vendored tree buildable as a standard
Linux ELF `llama-server` (plain cmake + g++/nvcc, rpath `libggml-cuda.so`,
clean gdb/nsys) alongside the untouched cosmocc Makefile lane.

Registry additions (all markers `opencoti-hook: native-elf-cmake (#619)`):

- `llama.cpp/ggml/src/CMakeLists.txt` — add `ggml-turbo-quant.c`,
  `ggml-neo-pipeline.cpp`, `ggml-iqk-flash-attn.cpp` to ggml-base sources.
- `llama.cpp/src/CMakeLists.txt` — add `dca.cpp` (matches neither the
  `llama-*.cpp` list nor the `models/*.cpp` glob).
- `llama.cpp/common/CMakeLists.txt` — add `pcie-profile.cpp`.
- `llama.cpp/ggml/src/ggml-cpu/CMakeLists.txt` — add
  `ggml-cpu/opencoti-elf-stubs.cpp` (CMake-lane-only).
- `llama.cpp/tools/server/CMakeLists.txt` —
  `set_source_files_properties(server.cpp … main=llama_server)`: the Mozilla
  overlay's server.cpp defines `int main` where upstream defines
  `llama_server`; the cosmo lane renames at compile time, CMake must too.
- NEW FILE `llama.cpp/ggml/src/ggml-cpu/opencoti-elf-stubs.cpp` — return-false
  stubs for the `llamafile/sgemm.cpp` dispatchers (`llamafile_mixmul`,
  `_mixmul_needs`, `_mixmul_iqk`, `llamafile_fa_*`); every call site treats
  false as "use the upstream generic path" (same contract as Mozilla's
  `fa_helpers_unsupported.cpp`), so the ELF build loses only CPU fastpaths.
  sgemm.cpp itself includes `<cosmo.h>` and stays cosmo-lane-only.

Bump guidance: on a future llamafile bump, re-anchor the five one-line list
insertions (trivial context) and keep the stub file as-is unless sgemm.cpp's
dispatcher signatures change. The CMake lane REQUIRES
`-DGGML_CUDA_FA_ALL_QUANTS=ON` — upstream's CMake only globs
`fattn-vec*.cu` (our turbo/TCQ instances + codebook setters) under that flag;
the cosmo build always sets it. CMake lane uses cuBLAS (not TinyBLAS) —
per-toolchain perf gating (bug-2103/2104 class).

Chain state after 0097: 43 patches, strict-apply + byte-identical replay
proven from clean 0.10.3 archive + Mozilla overlay on 2026-07-05.

## 2026-07-05 — 0098 rolling-KV lse-decode (bug-1843, #623)

`0098-rolling-kv-lse-decode.patch` extends the Stage-3a LSE channel
(`opencoti_fattn_dst_lse` / `opencoti_fattn_dst_lse_written`, both defined in
`fattn-common.cuh` and drained by `launch_fattn`) to `head_dim<=256` `n_q==1`
decode at the three streaming-FA call sites in
`llama.cpp/ggml/src/ggml-cuda/fattn.cu` (markers: `opencoti-hook:
f5-rolling-kv bug-1843` at the S3b overlap site; short `bug-1843:` notes at
the S3a single-stream and S2 resident-view sites). The per-tile
`streaming_lse_kernel` recompute now fires only when
`opencoti_fattn_dst_lse_written` reports that no finalize (stream-k fixup /
parallel-blocks combine) ran. No header change; no new symbols; DSO-only
rebuild. Chain state after 0098: 44 patches, strict-apply + byte-identical
replay proven from clean 0.10.3 + Mozilla overlay (2026-07-05).

## 2026-07-06 — 0099-0102 DCA-Gemma wiring + attn-rot inherit + streaming scratch ring (bug-2118/2119/2120/2121)

Four fixes captured after the Gemma×DCA contamination discovery (bug-2118) and
the two criticals it surfaced:

- `0099-dca-gemma4-wiring.patch` — surgical hooks in `llama.cpp/src/models/gemma4.cpp`
  + `gemma4-assistant.cpp`: `build_attn_inp_dca` + per-layer `dca_l` dispatch to
  `build_attn_dca` (GLOBAL layers; assistant Q-only). Marker: `opencoti F5 dca`.
- `0100-dca-single-chunk-defer.patch` — `llama.cpp/src/dca.cpp` (opencoti-owned file):
  single-chunk regime defers quant lift like analytical-bands.
- `0101-kv-attn-rot-inherit.patch` — `llama.cpp/src/llama-kv-cache.cpp` ctor share
  branch inherits attn_rot_k/v + Hadamard tables from the source cache. Marker
  comment: `opencoti bug-2120`.
- `0102-fattn-bulk-h2d-scratch-ring.patch` — `llama.cpp/ggml/src/ggml-cuda/fattn.cu`:
  bug-2116 bulk-H2D quant tail lift + bug-2121 persistent per-slot staging scratch
  ring (markers: `opencoti-hook: f5-rolling-kv bug-2116` / `bug-2121`). RULE banked in
  cerebrum: never share ggml_cuda_pool scratch across CUDA streams.

Chain state after 0102: 48 patches; full chain strict-applies from clean 0.10.3
pin (7fca8b2 + nested dbe9c0c8 + Mozilla overlay) and reproduces the live tree
byte-identically on all touched files (verified 2026-07-06, capture-b21xx).

## 2026-07-07 — 0103-0104 rolling-KV CPU-FA probe guard + q6_0/q4_0 FA-VEC cell (bug-2125 / #627)

Two fixes for the DCA + custom-KV boot SIGSEGV / abort found on bs2 at 256k/512k:

- `0103-kv-cpu-fa-probe-guard.patch` — `llama.cpp/src/llama-kv-cache.cpp`: gates
  the Stage-3d-S2 tactic selector's synthetic **CPU** flash-attn companion probe
  (`opencoti_fa_compute_probe_ms` with `on_cpu=true`) behind a new host predicate
  `opencoti_cpu_fa_capable_ktype`. That probe modeled an unused CPU_FA_TAIL cost;
  for CUDA-FA-VEC-only KV types (q6_0, turbo{2,3,4,8}_0, turbo{2,3}_tcq) the CPU
  vec_dot trait is NULL, so building/running the host flash-attn NULL-derefed →
  SIGSEGV at context init (before any request). The predicate returns 0.0f for
  custom-K tiers, leaving CPU_FA_TAIL unmodeled — the selector keeps its pure-GPU
  POSITION_WINDOW default (**no** CPU fallback: the rolling window stays fully on
  GPU for every KV tier). Marker comment: `opencoti bug-2125`. Host-only.
- `0104-fattn-q6q4-vec-instance.patch` — `llama.cpp/ggml/src/ggml-cuda/fattn.cu`
  + new `template-instances/fattn-vec-instance-q6_0-q4_0.cu` + `llamafile/build-functions.sh`:
  adds the q6_0-K / q4_0-V FA-VEC cell (strong-K, cheap-V — the K precision ≥ V
  invariant) missing from the dispatch matrix, which crashed the D≤256 path
  (Gemma-4 local/SWA) into `GGML_ABORT`. Marker: `opencoti-hook: q6_0 KV (#627/bug-2125)`.
  The reverse q4_0/q6_0 direction is intentionally NOT added (K must stay
  higher-bit). CUDA DSO change. Mirrors the q6_0-q5_0 pattern (#512).

Chain state after 0104: 50 patches; full chain 0001..0104 strict-applies from
clean 0.10.3 pin and reproduces the live tree byte-identically on all four
touched files (verified 2026-07-07, capture-b2125). Gates (bs2 256k DCA-on):
q6_0 / turbo3_tcq / turbo2_tcq symmetric + q6_0-K/q4_0-V all boot HEALTHY +
niah_single_1 = 100.0 (every one previously SIGSEGV/abort at boot).

## 2026-07-09 — 0105-0107 WS1 asym scalar-K/turbo-V ship (bug-2126/2127/2130, #628/#630)

**No new opencode source markers.** WorkStream-1 (asymmetric strong scalar-K +
cheap turbo/TCQ-V decode) lands entirely in the **vendored** `llama.cpp` tree as
three additive patches; the packages/ Registry above is unchanged. Split:
`0105` DCA per-tensor defer-decouple (bug-2127, dca.cpp, host-only, the ~2×
decode win); `0106` scalar-K/turbo-V boot-correctness guards (bug-2126,
llama-kv-cache.cpp, host-only, two null-fn-ptr SIGSEGV fixes in the rolling-KV
boot probe); `0107` turbo-MMA verify kernels (D128/D256/D512, OPT-IN default-OFF)
+ the 16 fused-VEC scalar-K/turbo-V decode instances + fattn.cu selector +
llama-graph.cpp fuse path + bug-2130 `need_f16` (fattn-mma-f16.cuh) +
build-functions.sh default-build inclusion. The dropped WS1 option-2 boot
"fuse-autoselect" scaffolding (`opencoti_turbo_fuse_autoselect`,
`g_opencoti_asym_fuse_enable[2][2]`, `probe_nq`/`materialize` probe params) is
**not** in any patch — it was stripped at ship (inert, default-fuse-equivalent).
The WS1 `// opencoti-hook: WS1 …` markers live in vendored source only (not
counted by the packages/ conformance grep). Per-file registry: the
`vendors/patches/llamafile/README.md` `0105`/`0106`/`0107` rows.

turbo-MMA VERDICT (#630, 3-model crossover A4B/omni27/qw35a3): earns no
default-on anywhere → shipped OPT-IN default-OFF (Gemma A4B/D256 wins ≤128k,
net-loses @256k + un-gatable windowed; Qwen/D128 bit-exact but tps-flat,
FFN-bound). Chain state after 0107: 53 patches; full chain 0001..0107
strict-applies from clean 0.10.3 pin and reproduces the intended FINAL tree
byte-identically (6 modified + 19 new TUs match; llama-kv-cache.h returns to
baseline; zero scaffolding symbols) — verified 2026-07-09, capture-ws1-ship.

## 2026-07-10 — 0108-0109 WS2 turbo-V in-register DCA-decode ship (bug-2134/2140, #629)

**No new opencode source markers.** WorkStream-2 (read a rotated turboN V-cache
IN-REGISTER inside the multichunk DCA fused flash-attn kernel, killing the
per-forward whole-cache `cpy_turboN_f16` cast) lands entirely in the **vendored**
tree as two additive patches; the packages/ Registry above is unchanged. Split:

- `0108-ws2-dca-fused-turbo-v-inregister.patch` (bug-2134) — `llama.cpp/src/dca.cpp`
  (the WS2 in-register gate, now **DEFAULT-ON**; `WS2_DCA_TURBO_V=0` is the
  f16-lift escape; n_q≤`WS2_NQ_MAX`(16) gate keeps large-n_q PREFILL on the
  byte-identical f16-lift path) + `ggml/src/ggml-cuda/fattn-mma-f16.cuh`
  (D512+D256 turbo/tcq `dca_fused` instance macros + the TCQ `to_fp16_nc` assert
  exemption for the in-register-V case) + `ggml/src/ggml-cuda/fattn.cu` (D256
  `case 256` turbo2_0/turbo3_0/turbo2_tcq/turbo3_tcq V-type dispatch) +
  `template-instances/generate_cu_files.py` + the new turbo/turbo-D256/turbo2tcq-D256/
  turbo3tcq-D256 `dca_fused` TU shards. Markers: `opencoti-hook: WS2 turbo-V DCA
  (#629)` (host + kernel). bug-455 (dca_fused forces `Q_in_reg=false`) inherited
  by the turbo D256 path. CUDA DSO + host change.
- `0109-cuda-dso-2gb-fatbin-last.patch` (bug-2140) — `llamafile/build-functions.sh`
  + `llamafile/cuda.sh`: two build hooks. **`cuda-fatbin-last` (PRIMARY)** — the
  ~1.9GB `.nv_fatbin` (6 SM arches × ~260 cubins) sat physically between host
  `.text` and its `.bss`/`.got`, so every host device-stub PC32/PLT/TLSGD reloc
  spanned >2^31 → link truncation once the D256 tcq TUs pushed the image past 2GB.
  Root fix = force `.nv_fatbin` to the image END via an `INSERT AFTER .bss`
  orphan-section linker fragment (pure `-Wl`, no code-model change; fixes ALL
  reloc classes; the only fatbin ref left is a 64-bit data pointer). **`cuda-mcmodel-large`
  (OPT-IN, default `none`)** — `OPENCOTI_MCMODEL=large|medium|none` forces the large
  code model on host x86 (compile via nvcc `--compiler-options`, link via
  `-Xcompiler`) + `-Xlinker=--no-relax` for the prebuilt-CRT/`libcudart_static`
  GOTPCRELX; superseded by fatbin-last, kept as a documented escape. Device SASS
  untouched → zero kernel-perf cost (perf-regression A/B confirms parity). Markers:
  `opencoti-hook: cuda-fatbin-last` + `opencoti-hook: cuda-mcmodel-large`. Build-flag/
  linker change only. Per-file registry: the `vendors/patches/llamafile/README.md`
  `0108`/`0109` rows.

WS2 VERDICT (#629, 4-arch × 4-tier matrix @256k DCA-on): **default-ON everywhere** —
in-register turbo-V is a validated decode WIN of +68-95% (D512 A4B/31B, D256
omni27/qw35a3 × turbo2_0/turbo3_0/turbo2_tcq/turbo3_tcq), lossless by RULER niah
(100) + teacher-forced logit-equiv (real_frac 0 / ffa 1.0), incl. A4B MTP-on
n_q>1 verify (real_frac 0, mean_tv 2e-05). Opposite of the WS1 turbo-MMA null:
WS2 removes an O(n_kv) whole-cache cast that grows with ctx, so it wins on the
serving decode path. Chain state after 0109: 55 patches; full chain 0001..0109
strict-applies from clean 0.10.3 pin and reproduces the live WS2 tree
byte-identically — verified 2026-07-10, capture-ws2-ship.

## 2026-07-10 — 0110 WS2 asym-band NaN guard (bug-2141, #629/#626)

**No new opencode source markers.** The WS2 (0108) in-register turbo-V dca_fused
decode reads V rotated and de-rotates the graph OUTPUT via `wht_o`; the normalized
`O = VKQ / rowsum` is produced in the shared stream-k combine. Under DCA-on decode
the fused kernel walks INTER→SUCC→INTRA phases under one continuous online-softmax
and load-balances via the stock kbc stream-k decomposition, so the final
divide-by-rowsum lands in the shared stream-k fixup for seam blocks (and the
process_tile complete-tile write-back otherwise). A band-boundary phase with an
all-(−inf) mask gives a ZERO exp-sum (rowsum==0) → 0/0=NaN, which wht_o's 128-pt
butterfly smears across all 512 de-rotated dims → NaN logits → empty generation.
Surfaces ONLY on asymmetric turbo_tcq K≠V (the hi-K/cheap-V serving pattern):
t3tcqK/t2tcqV + t2tcqK/t3tcqV pre-fix niah 20 / 4-of-5 EMPTY @256k/8-chunk (already
100 @512k/16-chunk) — a composition no symmetric control exercises. Discriminator
`WS2_DCA_TURBO_V=0` (V→f16-lift, same turbo3 K-lift) recovers to niah 100 / 0-empty,
isolating the fault to the in-register-V + wht_o composition (NOT the K-lift/codebook
— all per-tensor type selection verified correct).

- `0110-ws2-asym-band-nan-guard.patch` (bug-2141) — guards the zero exp-sum divide
  at the three FA output-normalization sites feeding wht_o:
  `ggml/src/ggml-cuda/fattn-mma-f16.cuh` (`flash_attn_ext_f16_process_tile`
  complete-tile write-back) + `ggml/src/ggml-cuda/fattn-common.cuh`
  (`flash_attn_stream_k_fixup_uniform` + `flash_attn_stream_k_fixup_general`
  stream-k seam combines). Emits the identity 0 for a fully-masked row; the
  `rowsum!=0` branch runs the EXACT original division (not a reciprocal-multiply)
  → bit-identical for all symmetric turbo/TCQ + scalar tiers and all non-DCA
  callers of these shared kernels; the WS2 decode win is untouched (no-op on the
  hot path). Markers: `opencoti-hook: WS2 asym-band NaN guard (bug-2141)` (3 sites,
  kernel-only). CUDA DSO change; no host binary change. Per-file registry: the
  `vendors/patches/llamafile/README.md` `0110` row.

RE-GATE VERDICT (#632, fixed DSO f1c554d8): full asym turbo_tcq matrix RECOVERED —
t3tcqK/t2tcqV AND t2tcqK/t3tcqV at 256k AND 512k all niah **100.0 / 0-empty**
(pre-fix t3tcqK/t2tcqV @256k was 20 / 4-of-5 empty). Symmetric controls
(turbo2_tcq, turbo3_tcq @256k) hold **100 / 0-empty**. WS2 in-register decode win
preserved through the guard (A4B turbo2_0 @256k warm **~91.2 tps**, uncontended).
Off-path bit-identicality confirmed by the symmetric/scalar controls staying 100
plus the guard's rowsum!=0 branch running the identical division. Chain state after
0110: 56 patches; full chain 0001..0110 strict-applies from clean 0.10.3 pin and
reproduces the live tree byte-identically — verified 2026-07-10
(fattn-mma-f16.cuh 58cdae34, fattn-common.cuh f7af3a8d).

## 2026-07-10 — 0111-0113 rolling-KV spill made graceful (bug-2144/2145/2146, #635/636/637/631)

Three host-only additive patches that make rolling-KV overflow (POSITION_WINDOW KV
residency) actually graceful — the "spill fallback" now degrades on the PCIe/bytes
axis instead of cliffing. All validated on the same binary (#637, Qwen2.5-14B-1M Q8_0
@128k, bs2 RTX 6000, single-GPU, needle PRESENT every cell); the DSO is unchanged
(the bug-2144 kernel tail-read `src[7]/[8]` already shipped in the C2 DSO with 0108).
New markers live in vendored source only; the packages/ Registry is unchanged.

- `0111-dca-streaming-tail-bug2144.patch` (bug-2144) — `llama.cpp/src/dca.cpp`. DCA-on
  rolling-KV spill decode cliffed (~55×) because `build_attn_dca_core` used the
  whole-cache `get_k`/`get_v`, which for a spilling POSITION_WINDOW layer return the
  `ggml_concat(window, tail)` fallback (sync full-KV round-trip every step), bypassing
  the async streaming path. Fix gives the fused `dca_fused` kernel a two-region
  window+tail streaming path (window on GPU, pinned-host tail on `src[7]/[8]`,
  `op_params[5]=n_win`), gated `dca_window_decode` (FA ∧ small-n_q ∧ !single_chunk ∧
  `!get_is_fragmented()` ∧ POSITION_WINDOW active ∧ window active). **DEFAULT-ON**
  (escape `WS2_DCA_STREAM=0`); safe because the master flag only arms the path and the
  predicate enforces every precondition structurally. Markers:
  `opencoti-hook: DCA streaming-tail (bug-2144-C / Stage C)` (2 sites). Host-only.
- `0112-rolling-kv-phasegate-prefill-bug2145.patch` (bug-2145) —
  `llama.cpp/src/llama-graph.cpp`. The non-DCA streaming-window path has no batched
  large-n_q kernel → PREFILL ran decode-style over the window = flat ~325 tok/s cliff.
  Fix gates the window fast-path to decode/small-n_q (`WS2_NQ_MAX` d16) and routes
  large-n_q prefill to the amortized whole-cache concat + batched `build_attn_mha`
  (path B), at both dispatch sites. Markers:
  `opencoti-hook: rolling-KV phase-gate prefill (bug-2145)` (3 sites). Host-only.
- `0113-rolling-kv-reserve-fixpoint-bug2146.patch` (bug-2146) —
  `llama.cpp/src/llama-context.cpp`. The bug-1342 two-pass reserve measured the compute
  buffer against a degenerate `wc=256` window; with the 0111 streaming decode path that
  forces a maximal tail whose scratch inflated the measurement (~4088 vs real ~926 MiB
  → 3.16 GiB over-reserve, a fitting KV pushed to a 3168 MiB tail). Fix iterates the
  reserve to a fixpoint (cap 3 iters) — each re-size grows the window/shrinks the
  tail/shrinks the buffer, converging to full residency when the KV fits; non-streaming
  converges in one step byte-identical to prior. No new marker (the fixpoint loop is
  self-documented `bug-2146`). Host-only.

VALIDATED (#637): DCA-on streaming spill tracks-and-beats the DCA-off rolling-KV
reference at every matched tail — plateau ~37 tps to a ~4.9 GiB tail, common knee
~5.6 GiB, graceful rolloff, NO cliff; ~17× recovery at a 6.7 GiB tail (0.68→11.69).
Reserve fixpoint recovers full residency (37.10→38.40 @vt=41888) and improves the
spill case (+62% @vt=35400). Default-on re-gate (no env) reproduces the numbers.
Chain state after 0113: 59 patches; full chain 0001..0113 strict-applies from clean
0.10.3 pin and reproduces the live tree byte-identically — verified 2026-07-10
(dca.cpp efa73317, llama-graph.cpp 874fa901, llama-context.cpp e40aa9e2). Per-file
registry: the `vendors/patches/llamafile/README.md` `0111`/`0112`/`0113` rows.

## 2026-07-10 — 0114 hybrid rolling-KV residency + 0111 ggml host-op amendment (bug-2142, #633)

- `0114-hybrid-rolling-kv-residency-bug2142.patch` (bug-2142) —
  `llama.cpp/src/llama-memory-hybrid.{h,cpp}`, `llama-memory-hybrid-iswa.{h,cpp}`,
  `llama-model.cpp`. The rolling-KV residency knobs (`--vram-target`,
  `--kv-residency-mode`, the bug-1342 compute-reserve, the window-measure-pass) reached
  the iSWA and plain-dense memory paths (llama-model.cpp `create_memory`) but were
  **silently dropped** for hybrid models: `llama_memory_hybrid` and
  `llama_memory_hybrid_iswa` hardcoded `0/0.0f/false` when building their internal
  attention `llama_kv_cache`, so a hybrid model (qw35a3 = Qwen3.6-35B-A3B DeltaNet+attn
  NextN MoE) on a too-small card ignored `--vram-target` and OOMed. Fix threads the five
  residency params through both hybrid wrapper ctors to the attention cache, mirroring
  the iSWA/plain forwarding; the new ctor params default to the prior off-state so any
  non-forwarding caller and every resident config stays byte-identical. Markers:
  `opencoti-hook: rolling-KV residency for hybrid (bug-2142)` (2 `create_memory` call
  sites in `llama-model.cpp`, one per hybrid variant). Host-only — no DSO change.
  VALIDATED (#633, bs2 GPU0): fixed `--vram-target 32000` @ `-c 262144` on qw35a3 →
  `host tail = 4705 MiB` (knob honored); pre-fix (C2ship) same cmd → `FULLY RESIDENT …
  no host tail` (dropped, GPU overshoots the 32000 target to 34907 MiB). Fully-resident
  decode unchanged (182.5 t/s); spill decode 77.8 t/s (bandwidth-bound, as expected).

- **0111 amendment (bug-2144 completeness).** The originally-shipped
  `0111-dca-streaming-tail-bug2144.patch` captured only the `dca.cpp` call-site
  (`kp_tail`/`vp_tail`/`dca_n_win`) but NOT the matching host-side op-node construction
  that call needs to compile — the `k_tail`/`v_tail`/`n_win` parameters on
  `ggml_flash_attn_ext_dca_fused` and their `src[7]`/`src[8]`/`op_params[5]` wiring in
  `ggml.h`/`ggml.c`. The shipped C2 binary built from a working tree that had those
  edits, so a strict re-apply of the chain would REVERT them and fail to compile; the
  gap was masked because the chain proof did not build. 0111 is regenerated to include
  the `ggml.h`/`ggml.c` hunks (recovered byte-identically from the pre-ship vendor
  backup). Lesson banked: the chain-apply proof MUST include a build step so a
  non-building chain can't hide an uncaptured edit. See buglog bug-2142/bug-2147.

Chain state after 0114: 60 patches. Full-chain 0001..0114 strict-apply + build proof
is the gating step for this section (in progress at commit time).

## 2026-07-11 — 0115 refuse context-shift on a spilled rolling-KV window (bug-2148, #638)

- `0115-rolling-kv-shift-guard-bug2148.patch` (bug-2148) —
  `llama.cpp/src/llama-kv-cache.cpp`, one hunk in `llama_kv_cache::get_can_shift()`.
  The F5 M7 Stage-3c position window splits the cache across a GPU-resident window
  `[0,wc)` and a CPU-resident tail `[wc,kv_size)` when `--vram-target` forces overflow.
  Context-shift (server `seq_rm`+`seq_add`→`pos_add`) and self-extend (`seq_div`→`pos_div`)
  re-rope the cache IN PLACE via the `llm_graph_input_k_shift` / `build_rope_shift` graph,
  which views only `k_per_stream` — sized to the window's `wc` cells, NOT `get_size()`
  (llama-kv-cache.cpp ~4720-4744). So the moment a shift fires with a spilled window it
  over-reads the window tensor and hits `ggml.c` `GGML_ASSERT(view_src == NULL || data_size
  == 0 || data_size + view_offs <= ggml_nbytes(view_src))`; and even were the view to fit,
  the host tail (written already-roped, never transformed again — no CPU-side rope path)
  would be left stale-roped. Fix: `get_can_shift()` returns `false` when any layer has an
  active spilled window (`window_cells > 0` && a populated `k_cpu_per_stream`); the server
  reads this at init (`common_init_from_params` → "KV cache shifting is not supported for
  this context, disabling KV cache shifting"; also server-context.cpp:1093) and cleanly
  disables `ctx_shift` + cache-reuse, so a spilled slot is bounded at `n_ctx` instead of
  crashing. Fully-resident windows (`wc==kv_size`, no tail, `window_cells` sentinel 0) and
  every non-window cache find no spilled layer → BYTE-IDENTICAL and still context-shift
  normally; iSWA (`get_can_shift` ANDs kv_base+kv_swa) and hybrid (`get_can_shift` delegates
  to `mem_attn`) wrappers propagate the leaf guard. The "no-degradation" alternative (a
  host-side rope-by-delta pass over the CPU tail on every shift) is DEFERRED — out of scope
  for the multi-session prefix-shared serving target, which does not rely on infinite-context
  shift over a spilled tail. Marker: `opencoti-hook: rolling-kv-shift-guard` (1 site).
  Host-only — no DSO change. VALIDATED (#638, bs2 GPU0, Qwen2.5-14B-1M-Q8_0, `-c 12288
  --context-shift --keep 256`): windowON (`--vram-target 18000`, 6912 resident / 5376 host
  tail) → shift disabled at init, request bounded (predicted_n=3840, 8448+3840=12288),
  coherent, NO assert; controlOFF (`--vram-target 80000`, fully resident) → shift STILL fires
  (n_discard=6015), coherent to 1219 — guard does not over-fire. Prior binary asserted at
  `ggml.c:1840` on the windowON shift. Byte-identical `git apply` of 0115 == the live edit.

Chain state after 0115: 61 patches.

## 2026-07-11 — 0116 auto KV-tier selection at boot (HAL #582 P1, #621)

- `0116-kv-auto-tier-hal582-p1.patch` (#621) — the HAL #582 P1 policy: on a dense
  full-attention model, auto-pick the LEAST-compressing scalar KV type pair that keeps the
  whole cache resident, rather than making the operator hand-tune `-ctk`/`-ctv`. New static
  helper `opencoti_auto_select_kv_tier` in `llama-kv-cache.cpp`, called in the
  `llama_kv_cache` ctor BEFORE any residency/allocation math consumes `type_k`/`type_v`. It
  reuses the EXACT budget arithmetic of `opencoti_compute_resident_window_cells` (free −
  already-resident − compute-reserve, capped by `--vram-target`) and the same per-layer
  per-cell byte sum, then walks the ladder `{f16, q8_0/q8_0, q8_0/q4_0, q5_1/q4_0}` (HAL §2
  hi-K/cheap-V recipe; `q5_1/q4_0` last-resort floor) and selects the first (least-
  compressing) rung whose whole-cache cost fits; floor if none fit (rolling-KV then spills the
  excess). Announced at WARN with a `set -ctk/-ctv to override` hint (mirrors the sparse-V
  auto-policy #567). **Strictly opt-in** (`OPENCOTI_KV_AUTO_TIER=1`) and a no-op — byte-
  identical — when the env is unset (every existing caller), there is no GPU offload, the user
  set `-ctk`/`-ctv` off the f16 default (override respected), the model is iSWA
  (`swa_type != NONE`, Gemma stays f16/resident per HAL §3c), or f16 already fits.
- Marker: `opencoti-hook: kv-auto-tier` (1 site — the ctor helper + call in
  `llama-kv-cache.cpp`). Provenance: opencoti-original.
- Host adapter opt-in: `kvAutoTier` config → `OPENCOTI_KV_AUTO_TIER` env in `buildServerArgs`
  (`packages/opencoti-llamafile`). Additive, no surgical hook.
- Host-only — no DSO change (make-only build, ~11 s ccache-warm). VALIDATED (#621, bs2 GPU0,
  Qwen2.5-14B-1M-Q8_0, `-c 32768`, RTX 6000): ladder walks correctly as `--vram-target`
  tightens — 20000 MiB → q8_0/q8_0 (KV 3264 MiB fits budget 3328), 19400 → q8_0/q4_0 (2496
  fits 2728), 18800 → q5_1/q4_0 (2016 fits 2128); every rung `FULLY RESIDENT … no host tail`,
  passkey needle clean. Config-equivalence: auto q8_0/q8_0 output byte-identical to explicit
  `-ctk q8_0 -ctv q8_0`, which emitted zero auto-tier WARN (opt-in path confirmed off).
  Byte-identical `git apply` of 0116 == the live edit.

Chain state after 0116: 62 patches.

## 2026-07-11 — 0117 re-rope a spilled rolling-KV window on context-shift (#639, bug-2148 reopened)

- `0117-rolling-kv-shift-rerope-639.patch` (#639) — supersedes 0115. 0115 stopped the
  spilled-window shift from crashing by REFUSING it (`get_can_shift()` returned false when a
  layer had an active spilled window), which cleanly but silently disabled `ctx_shift` AND KV
  prompt-cache-reuse for any multi-session slot whose KV spilled — the exact serving-path
  capability this fork exists to preserve. 0117 does the deferred hard part instead:
  `build_graph_shift` re-ropes BOTH regions of a spilled window with the same `build_rope_shift`
  op — the WINDOW on its GPU tensor `k_per_stream[s]` (a view bounded to `wc` cells, so it never
  over-reads `get_size()`; shift slice `[base, base+wc)`) and the TAIL on its host tensor
  `k_cpu_per_stream[s]` (a view of the `kv_size-wc` tail cells; shift slice `[base+wc,
  base+kv_size)`). The tail tensor lives in host memory, so ggml's scheduler dispatches its rope
  op to the CPU backend automatically — identical rope formula, one-shot on shift events only,
  never per decode token — while attention still runs entirely on GPU over the already-roped
  window+tail (no CPU-attention / CPU_FA_TAIL divergence class). `get_can_shift()` returns true
  again (keeping the STEP35 and `n_pos_per_embd()>1` guards); the 0115 refusal loop is removed.
- Regression shield: fully-resident windows (`window_cells` sentinel 0) and every non-window
  cache take the `windowed==false` path; for `n_stream==1` the window view is sized `get_size()`
  and passes `k_shift_src` directly — BYTE-IDENTICAL to the pre-0117 op (the `n_stream>1`
  non-windowed view is unchanged too). Host-only, no CUDA DSO rebuild (the tail rope is an
  existing ggml op dispatched to CPU).
- Markers: `opencoti-hook: rolling-kv-shift` — 2 sites (`get_can_shift`, `build_graph_shift`),
  RENAMED from 0115's single-site `rolling-kv-shift-guard`. The 0115 patch file and its historic
  marker text remain (0117 supersedes its behavior additively; 0115 stays in the chain).
- VALIDATED (#639, bs2 GPU0, clean/no-contention, Qwen2.5-14B-1M-Q8_0, `-c 12288 --context-shift
  --keep 256`): spilled (`--vram-target 18000`, 6912 resident / 5376 host tail = 1008 MiB) →
  shift ENABLED, coherent through the shift (ZULU-9999 present, monotonic run 1219, output
  byte-identical to the fully-resident control), NO assert. Decode-tps == pre-#639 baseline
  within noise (74.88→43.67 over ~2800 tok vs baseline 74.47→40.25); the re-rope is shift-event
  only, no per-token cost. The 26.9-tps "regression" seen in the first gate was GPU/host
  contention (bug-2149), not this patch. Byte-identical `git apply` of 0117 == the live edit.

Chain state after 0117: 63 patches.

## 2026-07-11 — 0118 T\*-aware auto rolling-KV spill (HAL #582 P1+, #621 follow-on)

- `0118-kv-auto-tier-tstar-spill.patch` (#621 follow-on) — extends the 0116 auto-tier helper
  `opencoti_auto_select_kv_tier` so it can KEEP a higher-quality tier (up to f16) with a small
  host-tail spill when that spill is provably cheap, instead of always quantizing to the first
  fully-resident rung. The ladder fit-condition changes from "first tier fully resident" to
  "first tier whose SPILL ≤ `T*_bytes`", `T*_bytes = (target_drop·t_resident − c_engage)·bw_eff`
  (target_drop default 0.20, env `OPENCOTI_KV_TSTAR_DROP`; clamp ≥0, ≤ cliff env
  `OPENCOTI_KV_TSTAR_MAX_SPILL_MIB` default 800). Boot-measured terms: T1 explicit-axis handling;
  T2 `bw_eff = min(pcie, host_ram)` with a NEW host 1-thread memcpy probe (`OPENCOTI_KV_HOST_RAM_GBPS`);
  T3 `fa_ms_cell`+`c_engage` from the reused FA compute microbench (#351, `OPENCOTI_KV_CENGAGE_MS`);
  T4 the drop-model. Fast-path guard keeps the "f16 fits" boot byte-identical; probe-unavailable or
  T*≤0 degrades exactly to 0116's fully-resident gate; the ctx-value gate (req #2) falls out of the
  formula (short ctx ⇒ tiny t_resident ⇒ c_engage dominates ⇒ quantize).
- Strictly opt-in (`OPENCOTI_KV_AUTO_TIER=1`, the same 0116 switch) and a byte-identical no-op when
  unset. HOST-ONLY: the FA microbench dispatches to the existing CUDA DSO — no ggml-cuda.so rebuild.
- Marker: `opencoti-hook: kv-auto-tier` — EXTENDS the existing 0116 site (the ctor helper +
  its one call), no new hook site. The 0116 marker text covers it.
- VALIDATED both hosts (T2-T4 terms fire; solidPC PCIe-bound bw_eff 6.7, bs2 host-RAM-bound 9.3-10.3;
  drop-model monotone in target_drop). T5 RULER-niah quality/tps gate PASS (bs2 GPU0,
  Qwen2.5-14B-1M-Q8_0, `-c 65536`, `--vram-target 27000` → ~2 GB f16 spill): auto → q8_0/q8_0
  (T* only ~14 MiB), f16-window-spill decode-tps drops 78.5% (≫ 20% target), q8 niah == f16 niah
  == 100 (lossless), q8 tps 35.3 = 3.3× f16-spill 10.8 — quantize verdict correct, fallback
  dominates. Byte-identical `git apply` of the full chain 0001..0118 == the live edit (cmp OK).

Chain state after 0118: 64 patches.

### `0119-p0-dcaon-scalar-inregister` (#582-P0 / #643 + #644)

Scalar q4_0/q4_1/q5_0/q5_1/q6_0/q8_0 K/V read IN-REGISTER inside the multichunk
`flash_attn_ext_f16_dca_fused` MMA kernel, killing the whole-cache `to_fp16_nc`
dequant-lift that dominated quant-KV DCA-on decode (HAL §3b profile ~29.5 s/decode).
Scalar generalization of #629's turbo-V in-register read; reuses the SAME
`load_tile_dequant` / `get_dequantize_V` path (no new kernel math).

- Marker: `opencoti-hook: P0 DCA-on scalar in-register` — 3 sites: `fattn-common.cuh`
  (the 4 new `ne==8` V-reader branches, q4_1/q5_0/q5_1/q6_0 — the only correctness
  surface, bug-2152; the MMA loader stays fixed at `ne_per_chunk=8`, SHARED with the
  proven turbo path, NOT changed); `fattn-mma-f16.cuh` (skip in-launch K materialize
  for a scalar `type_K`, mirroring the existing `type_V==F16` guard; + D128 scalar
  externs); `fattn.cu` (decode-gated dispatch, `Q->ne[1] ≤ WS2_NQ_MAX`=16, env
  `WS2_DCA_SCALAR_KV` default on — matched pair dispatches its typed instance, any
  other combo falls through to the shipped F16-materialize default).
- 40 NEW template-instance TUs (`fattn-mma-f16-dca-fused-scalar-<tag>-d128-{0..3}.cu`;
  4 ncols2 shards × 10 pairs) — named `fattn-mma-*.cu` so `collect_gpu_sources` globs
  them. D128 only (Qwen full-attention); D256/D512 scalar deferred (Gemma KV is turbo,
  already in-register).
- Correctness: #643 beachhead q8_0-K/q4_0-V @D128 ctx128k `real_frac 0.0` + 2.13×
  decode (14.72→31.33 tps); #644 all 10 pairs @D128 32k multichunk DCA-on `real_frac
  0.0`. Byte-identical `git apply` of 0119 onto a reconstructed 0118 baseline == the
  live edit (cmp OK, 43 files). Additive-only, touches only nested llama.cpp files (no
  outer-file conflicts). DSO 2.5→3.83 GB, absorbed by the fatbin-last fragment
  (bug-2140); `-mcmodel=large` stays OFF. **Applies after `0118`.**

Chain state after 0119: 65 patches.

### `0120-p2-mixed-kv` (#622 / #582-P2)

Position-axis **mixed KV** for rolling-KV spill: the resident window `[0,wc)` keeps
the boot KV type (e.g. `q8_0`) while the spilled host tail `[wc,n_kv)` may carry a
MORE-compressed type (e.g. `q4_0`). Recent (attention-dominant) tokens stay
high-fidelity on-device; the older tail pays fewer H2D bytes per decode step.
Position-axis complement to the head-axis (#620/#643 P0) and boot-tier (#621 P1)
residency levers. New CLI `-ctkt`/`-ctvt` set the tail K/V type; sentinel
`GGML_TYPE_COUNT` (flag unset) means "tail == window" → every path is byte-identical
to the shipped uniform-window behaviour.

- Marker: `opencoti-hook: P2 mixed-kv` — additive 5-point plumbing carrying the tail
  type from CLI to the pinned host allocation: `common/arg.cpp`
  (`--cache-type-k-tail`/`-v-tail` → `common_params.cache_type_k_tail`/`_v_tail`,
  `common/common.{h,cpp}`) → `llama_context_params` (`include/llama.h`) →
  `llama_cparams` (`src/llama-cparams.h`) → kv-cache ctor
  (`src/llama-kv-cache.{cpp,h}`, `src/llama-kv-cache-iswa.{cpp,h}`,
  `src/llama-context.cpp`, `src/llama-model.cpp`), which allocates
  `k_cpu_per_stream`/`v_cpu_per_stream` at the tail type. **Decode** (the win): the
  POSITION_WINDOW streaming FA op (`ggml/src/ggml-cuda/fattn.cu`) already reads
  window + tail each in-register at its own type (reuses #620/#629 readers) — no
  whole-cache f16 materialise.
- Two correctness-fallback fixes in `src/llama-kv-cache.cpp`, both host-only, both
  byte-identical when tail type == window type:
  - **bug-2161** — the get_k/get_v concat-reassembly accessor (prefill / graph-reserve,
    n_q > `WS2_NQ_MAX`) feeds `ggml_concat`, which asserts `a->type == b->type`. Fixed
    with a file-local `poswin_lift_to_f16` (mirror of `dca_lift_to_f16`: f16
    passthrough; scalar quants via `ggml_cast`→F32→F16, byte-exact GPU CPY) applied to
    both region views only when the two types differ.
  - **bug-2162** — the Stage 3c-6 state-IO window paths (K/V × write/read) used the
    window row size for the tail region, so an in-memory context-checkpoint
    (llama.cpp PR16391, on by default) asserted on the 2nd prompt at a mixed config.
    Fixed with a per-region `ggml_row_size(k_cpu->type / v_cpu->type, n_embd_*_gqa)`,
    symmetric across writer and reader; parity assert recomputed from both region sizes.
- Gate (S7, bs2 Qwen2.5-14B-Instruct-1M-Q8_0, `-c 24576` `--vram-target 18000`,
  logit-equiv vs full-resident gold; NEVER greedy needle): **correctness** `real_frac
  0.0` for U8/MIX/U4, advisory mean-TV **U8 0.0069 < MIX 0.0110 < U4 0.0118** (mixed
  strictly between uniform-q8 and uniform-q4, closer to gold than uniform-q4);
  **throughput** MIX 28.1 vs U8 26.4 tps = **+6.4%** (host tail 11264 cells);
  **regression** `-ctkt q8_0 -ctvt q8_0` sentinel byte-identical to unset (REG==U8).
  The gate exercises context checkpoints, so it doubles as a checkpoint round-trip test
  for bug-2162. Byte-identical `git apply` of 0120 onto a reconstructed 0119 baseline ==
  the live edit (0 files differ) + clean reverse-apply on the live tree. Additive-only,
  touches only nested llama.cpp files. **Applies after `0119`.**

Chain state after 0120: 66 patches.

### `0121-p2-auto-tail` (#653 / #582-P2)

**P2-auto boot policy**: when the P1 boot auto-tier (`opencoti_auto_select_kv_tier`,
0116/0118) is FORCED to spill — the auto-selected KV window still overflows the VRAM
budget, so the rolling-KV path spills a tail regardless — auto-pick a MORE-compressed
type for the spilled tail `[wc,n_kv)` than the resident window, reclaiming H2D bytes
on the oldest (least-attended) cells for free. Reuses the 0120 tail plumbing
(`type_k_tail`/`type_v_tail`, `GGML_TYPE_COUNT` sentinel = "tail == window"); this
patch only adds the boot POLICY that fills those sentinels in.

- **Opt-in is SEPARATE from P1**: `OPENCOTI_KV_AUTO_TIER_TAIL=1` (in addition to
  `OPENCOTI_KV_AUTO_TIER=1`). With P1 alone the tail stays == window, so a P1-only
  boot is byte-identical to before this patch — the compressed tail engages only when
  the operator also asks for it. An explicit `-ctkt`/`-ctvt` (tail != COUNT) is honored
  and never overridden, mirroring how `-ctk`/`-ctv` block the window auto-tier.
- **Policy — "q4_0 floor when window > q4_0"**: for each axis independently, if the
  chosen window type is higher-bit than q4_0 (`f32/f16/bf16/q8_0/q6_0/q5_1/q5_0`) and
  that axis's tail is unset, set the tail to q4_0; otherwise leave tail == window (so a
  window already at/below q4_0 is not "upgraded" to a coarser tail). The policy hook
  (`maybe_auto_tail`) fires ONLY on the two spill exits — the "window kept but spills"
  T\* branch (chosen==0) and the "chosen tier still spills" tail — so it never touches
  the fully-resident fast path (early return).
- Marker: reuses the existing 0116 `opencoti-hook: kv-auto-tier` site — this edit lives
  inside `opencoti_auto_select_kv_tier` and its one call site, exactly like 0118. No new
  inline hook site; the 0116 marker text covers it.
- Gate (bs2, Qwen2.5-14B-Instruct-1M-Q8_0, `-c 24576` `--vram-target 18000`,
  `--flash-attn on`, logit-equiv vs full-resident gold; NEVER greedy needle): **P2AUTO**
  (both envs) boot log shows `auto KV tail = q4_0/q4_0` over a real ~189 MiB / 3072-cell
  spilled region (window q5_1/q4_0 at the second-pass binding budget — bug-1342 two-pass
  compute reserve runs the auto-tier twice, the first looser pass fits, the binding pass
  spills), `real_frac 0.0`, advisory mean-TV 0.0087 (not worse than the uniform-tail
  P1AUTO floor). **P1AUTO** (`OPENCOTI_KV_AUTO_TIER=1` only) shows NO tail line —
  regression shield: byte-identical to the shipped P1 boot. HOST-ONLY (`src/llama-kv-cache.cpp`
  is libllama in the host binary, not the CUDA DSO): `make` rebuild, DSO ccache-hits.
- Byte-identical `git apply` of 0121 onto a reconstructed 0120 baseline == the live edit
  (cmp OK) + clean reverse-apply. Full chain 0001..0121 strict-applies from clean 0.10.3.
  Additive-only, touches only the nested `llama.cpp/src/llama-kv-cache.cpp`. **Applies
  after `0120`.**

Chain state after 0121: 67 patches.

## 2026-07-13 — 0122 RYS layer duplication `--repeat-layers` (#656–#668)

### `0122-rys-layer-duplication` (RYS-1..12)

**`--repeat-layers SPEC`** — load-time, weight-shared re-run of source transformer
layers *in place* (RYS / dnhkng "passthrough self-merge" — mergekit passthrough +
SOLAR depth-upscaling, done at inference time with no new weights). SPEC =
`;`-separated `i,j` blocks re-running layers `[i,j)` a second time right after the
original run; the **empty** plan ⇒ off ⇒ **byte-identical** to before this patch (eff
== src, identity map). All host-side; **no CUDA**.

- **Data plane.** `common_params.repeat_layers` (parsed in `common/arg.cpp`) →
  `cparams.layer_plan`, a `std::vector<uint32_t>` of length `n_layer_eff` where
  `layer_plan[eff] = src` (the original layer each effective step re-runs). Parsed +
  validated in `src/llama-context.cpp` (`opencoti_parse_repeat_layers`): fail-loud on
  out-of-range or malformed spec. The forward graph iterates the EFFECTIVE plan and
  passes `eff` (the KV slot) to `build_attn`; **weights/hparams are keyed by
  `src = layer_plan[eff]`** (RYS off ⇒ eff==src ⇒ every existing lookup is identity).
- **KV allocation across all cache families.** `map_layer_ids` + slot allocation
  extended to `n_layer_eff` in: the **unified** cache (`llama-kv-cache.{cpp,h}`), the
  **iSWA dual-cache** (`llama-kv-cache-iswa.{cpp,h}` — base + swa each eff-length, the
  `is_swa` filter resolved via an inner eff→src so a duplicated global stays global),
  the **recurrent/hybrid** state caches (`llama-memory-recurrent.{cpp,h}`,
  `llama-memory-hybrid{,-iswa}.{cpp,h}`), and the **DCA** build path (`dca.cpp` — eff→src
  for the single per-source `is_swa` hparam it reads).
- **Per-arch forward loops** (`src/models/*.cpp`): Qwen2/2.5/3 dense, Qwen2-MoE +
  Qwen3-MoE, Qwen3.5-MoE + Qwen3-Next recurrent (gated-delta-net), and Gemma-4 iSWA
  dense (12B/31B, per-layer `√` embed-scale keyed by src) + A4B (`gemma4.cpp`). Bucket-C
  recurrent boot crash root-caused + fixed here (auto-FA/GDN device resolvers parsed the
  EFFECTIVE idx from the op name → `dev_layer(eff)` out-of-range; eff→src map at 3 sites,
  bug-2160).
- **Boundary-layer advisory** (bug-2164): a boot `LLAMA_LOG_WARN` when the plan
  duplicates a layer within the first/last `max(3, n_layer/8)`. Duplicating early/late
  layers commonly yields incoherent output — this is a **MODEL** property (franken-merge
  boundary fragility), **NOT an engine bug**: verified by a middle/boundary × SWA/global
  2×2 on native Gemma-4-A4B via the CHAT endpoint (middle SWA L4/L10 + middle GLOBAL
  coherent; boundary GLOBAL L29 garbage — garbage tracks boundary-ness, independent of
  SWA/global). Coherent band ≈ L4..L24 (Gemma-4-A4B), only the last layer on Qwen3-8B.
  The advisory is the entire "fix"; there is **no correctness code change** — the engine
  is provably correct on middle layers.
- **Markers**: `opencoti-hook: rys-layer-dup` (vendored-source markers only — additive
  vendored edits registered here; do **not** change the `git grep "opencoti-hook:"
  packages/ docs/` upstream-opencode inventory). Provenance: opencoti-original (adopts
  dnhkng RYS-II + mergekit-passthrough conventions).
- **Gates** — coherence smoke via the **CHAT** endpoint (RYS *intentionally* changes the
  model, so logit-equiv-vs-baseline is the wrong bar; and the target models are
  thinking-models whose answer lives in `reasoning_content` — raw greedy `/completion`
  is garbage even at baseline). Qwen3-8B dense off/plain/DCA/vram-window/sparse all
  coherent; Gemma-4 A4B (MoE-iSWA) + 31B (dense-iSWA) middle-layer dup coherent, boundary
  dup garbage + advisory fires; RYS × rolling-KV residency window × DCA-on composes
  coherently on iSWA (window engages, no boot crash); recurrent (qwen35moe GDN) boots +
  self-stacks including full-model `0,40`.
- Byte-identical `git apply` of 0122 onto a reconstructed 0121 baseline == the live edit;
  full chain 0001..0122 strict-applies from clean 0.10.3. TS adapter: `repeatLayers`
  field/flag in `packages/opencoti-llamafile/src/{config,launch}.ts`. **Applies after
  `0121`.** See [features/rys_layer_duplication.md](../features/rys_layer_duplication.md).

Chain state after 0122: 68 patches.

## 2026-07-13 — 0123 RYS template probe `--rys-probe` (#669–#670)

### `0123-rys-probe` (rys-probe v1.1)

**`--rys-probe`** — a new host-only PROGRAM MODE (`ProgramMode::PROBE`) that maps a
GGUF for good RYS layer-duplication templates (companion to the 0122 `--repeat-layers`
engine), the cheap alternative to dnhkng RYS-II's days-long beam + XGBoost-surrogate +
graded-benchmark search.

- **New standalone file** `llamafile/rys_probe.cpp` (driver, `lf::rys_probe_main`), detected
  in `llamafile/args.cpp` (→ `ProgramMode::PROBE`, enum in `args.h`), dispatched from
  `llamafile/main.cpp`, built via `llamafile/BUILD.mk`. Four additive `common_params` fields
  (`rys_probe`, `rys_probe_widths="auto"`, `rys_probe_band="auto"`, `rys_probe_topk=10`) +
  four `--rys-probe*` flags in `llama.cpp/common/{common.h,arg.cpp}`.
- **Mechanism**: parse args as the PERPLEXITY example, disable `-fit` (its trial context
  aborted the flash-attn print), `common_init_from_params(model_only=true)` — load ONCE.
  For each contiguous `[i,j)` in the band `[max(3,n/8) .. n-max(3,n/8))` × widths, rebuild
  ONLY the `llama_context` (`llama_init_from_model`, `cp.repeat_layers="i,j"`; layer_plan has
  no runtime setter, weights are plan-independent), score **ΔPPL** (small corpus, compact
  batched NLL) + a **graded 24-item task-probe** (greedy + substring early-stop), free ctx.
- **Output**: ranked table + **exactly two templates** — MOST EFFICIENT (max ΔPPL/added-layer)
  + MAX GAIN (lowest ppl); honest "same block is both" when no wider block wins. The `task`
  score is both a rank signal and a guardrail (drop candidates below base `task`). The two
  templates feed `perf/llamafile/rys-probe-eval.sh` for the real downstream verdict.
- **Markers**: none needed — this is a NEW standalone mode, not an invisible behavior change
  to an upstream file; a run without `--rys-probe` is byte-identical (the model path is
  untouched). Provenance: opencoti-original (cheap analogue of dnhkng RYS-II + mergekit
  passthrough conventions). No TS adapter (a diagnostic CLI, not a runtime server flag).
- **Gates**: host builds clean (incremental; the `common_params` field-ADD needs a clean
  `rm -rf o` rebuild — bug-2160/2166 ABI-skew class — but the later widths default-VALUE
  change is ABI-safe). Smoke on Qwen3-4B-Q4_K_M (band [10,16) auto, 14 cand): base ppl 30.15
  task 0.917 → winner 11,13 (ppl 28.03, task 0.958). Mapped Qwen3.6-35B-A3B-UD-IQ3_XXS
  (n_layer 40, band [5,35), 109 cand) on the 3090.
- Byte-identical `git apply --reverse --check` of 0123 against the live tree (hunks match
  exactly); `common.h`/`arg.cpp` baseline reconstructed from the post-0122 pre-rys-probe
  vendor-backup, the 5 outer `llamafile/*` files diffed against HEAD (== post-0122; no prior
  patch touches them). Full-chain 0001..0123 strict-apply from clean 0.10.3 = Phase B (post-map,
  via `build:llamafile:check`). **Applies after `0122`.** See
  [features/rys_probe.md](../features/rys_probe.md).

Chain state after 0123: 69 patches.

## 2026-07-13 — 0124 RYS×MTP draft-context guard (bug-2171)

### `0124-rys-mtp-ctx-guard` (bug-2171)

**RYS `--repeat-layers` must not propagate into an MTP/NextN draft context.** With RYS
active *and* self-speculation on (`--spec-type draft-mtp`, Qwen NextN or the gemma4
assistant), the draft `llama_context` inherited the target's `params.repeat_layers` (it
is built from `common_context_params_to_llama(params_base)`), so its `cparams.layer_plan`
was RYS-duplicated too. But the NextN head sits at the canonical `il = n_transformer`
(`n_layer - nextn_predict_layers`), which the RYS eff→src map never contains → boot aborts
at draft-context init with `cpy_k: map_layer_ids MISS il=n_transformer (n_keys=0)` →
`failed to create MTP context`. RYS is a **target-only** inference capability: in self-spec
the NextN head consumes the target's *already-RYS* hidden state through shared memory, so
the drafter must run the plain base layer stack.

- **Fix** (`src/llama-context.cpp`, in the RYS plan-expansion block of the ctor): gate the
  spec on context type —
  `const char * rys_spec = (cparams.ctx_type == LLAMA_CONTEXT_TYPE_MTP) ? nullptr : params.repeat_layers;`
  then parse that (nullptr ⇒ empty plan ⇒ eff==src identity for the draft). The **target**
  context still receives `params.repeat_layers` unchanged, so RYS on the target is
  untouched; only the draft context is forced plain. `sched_reserve()` mirrors the same
  guard so the reserve graph matches.
- **Lossless**: greedy self-spec has the target verify every drafted token, so the
  drafter running the plain stack (vs an RYS-duplicated one) changes only acceptance/tps,
  never output. Boot now clean: RYS `5,9` + `draft-mtp` on Qwen3.6-35B-A3B boots with
  `RYS … 44 effective layers` **and** `ctx_dft=yes` (no MISS).
- **Byte-identical when RYS is off** (empty `repeat_layers` ⇒ the guard picks nullptr
  either way) and when MTP is off (target path unchanged). Marker: reuses the 0122
  `opencoti-hook: rys-layer-dup` site (same feature, no new hook). HOST-ONLY (no CUDA).
  Provenance: opencoti-original. **Applies after `0123`.**
  See [features/rys_layer_duplication.md](../features/rys_layer_duplication.md).

Chain state after 0124: 70 patches.

## 2026-07-14 — 0125 hybrid order-agnostic rolling-KV window (bug-2172 / #672)

### `0125-hybrid-window-orderagnostic` (bug-2172 / #672)

**Order-agnostic rolling-KV position-window scatter for a HYBRID model's attention
sub-cache.** On qwen35moe / qwen3next (gated-delta-net recurrent + full-attention + MoE +
NextN), the hybrid memory is `llama_memory_hybrid` wrapping a `llama_kv_cache` (`mem_attn`,
`!is_recurrent`) and a `llama_memory_recurrent` (`mem_recr`). When rolling-KV arms its
position-window on `mem_attn` under tight VRAM / large ctx, the recurrent batch pipeline
(`split_seq` + `[TAG_RECURRENT_ROLLBACK_SPLITS]` + `find_slot` cell-reuse/wrap) hands the
attention scatter **non-monotone destination cell idxs**, which trip the append-only window
scatter's monotone `GGML_ASSERT` in `cpy_k` (`llama-kv-cache.cpp:4083`, pre-patch).

- **Fix** (host-only; single TU `src/llama-kv-cache.cpp` + a one-line setter in
  `src/llama-kv-cache.h` + a one-line call in `src/llama-memory-hybrid.cpp`): a new
  `llama_kv_cache::append_only` flag (default **true**) is set **false** only by
  `llama_memory_hybrid` on its `mem_attn`. Every dense/iSWA/unified cache keeps
  `append_only=true` → the append-only fast path is **byte-identical + unchanged**. When
  false **and** in window mode:
  1. the resident window (`k/v_per_stream`) and host tail (`k/v_cpu_per_stream`) each get
     **+1 "trash" sink cell** (`wc_alloc` / `tail_c_alloc`); the append-only path never
     touches it (reads/writes only `[0,wc)` / `[0,tail_c)`), so its compute is byte-identical;
  2. `build_input_k/v_idxs` emit a **2-row `[n_tokens,2]`** routing tensor (row 0 =
     window-route → dest cell or window-trash; row 1 = tail-route → dest−wc or tail-trash),
     filled by `set_input_k/v_idxs`. `ne[0]` stays `n_tokens` so every `->ne[0]==n_tokens`
     graph guard holds;
  3. `cpy_k` / `cpy_v` scatter the **full per-stream K/V into BOTH** window and tail via two
     arbitrary-dest `ggml_set_rows` (non-sorted dests are legal → the monotone assert is
     gone), off-target copies landing in the never-read trash cell. **Same op-count at reserve
     and live** (exactly 2 set_rows/stream) so the reserve topology matches.
- **Read side UNCHANGED**: `get_k/v_window` reads `[0,min(n_kv,wc))`, `get_k/v_tail` reads
  `[0,n_kv−wc)` — these already cover every active scattered cell, and the streaming-FA
  `KQ_mask` `-INF`-masks the rollback holes by absolute key position. **Mask contract
  verified**: `GGML_OP_STREAMING_FLASH_ATTN` is fully mask-driven, per-cell,
  absolute-key-position-indexed, softmax-denominator-normalized → **zero CUDA change, no DSO
  rebuild** (host `make` only).
- **Scope**: iSWA-hybrid is out of scope (no current model is both hybrid AND SWA; the window
  never arms on `swa_type!=NONE`). Documented as such.
- **VALIDATED** (solidPC host `make`): qwen35moe auto-spill boots with NO assert + correct
  output; dense (Qwen) + iSWA (Gemma) regression shields byte-identical (append_only path
  untouched). Snapshot-diff capture (**3 files**) from a reconstructed post-0124 baseline;
  **byte-identical-verified** (`git apply` onto 0124-state == live, all 3 files sha256-match).
  Marker: `opencoti-hook: hybrid-window-orderagnostic` (**3 sites**: the `append_only` flag +
  the `cpy_k` / `cpy_v` scatter branches). HOST-ONLY (no CUDA). Provenance: opencoti-original.
  **Applies after `0124`.** See [features/rolling_kv.md](../features/rolling_kv.md).

Chain state after 0125: 71 patches.

### `0126-p0-dcaoff-scalar-mma-decode` (#582-P0 / #620, #644-mirror)

**DCA-off scalar-quant MMA decode.** Single-token quant-KV decode with DCA OFF was stuck
on the FA-VEC reader, which HAL §3b measured as kernel-bound (~2× f16 regardless of bytes
moved — the read never became bandwidth-bound). f16 at high-GQA / long-KV already falls to
the tensor-core MMA path (the `fattn.cu` selector); this patch gives a scalar-quant K/V
cache the SAME route: it reads the raw quant blocks straight into the MMA smem tile
in-register (`flash_attn_ext_f16_load_tile_dequant` / `get_dequantize_V` — no whole-KV
`to_fp16_nc` materialise) and runs the tensor-core `mul_mat`. Quant decode therefore
becomes bandwidth-bound and BEATS f16 (fewer KV bytes moved). This is the **DCA-off
mirror** of `0119`'s DCA-on scalar in-register read; the two share the same in-register
reader family (#629/#630).

- **--fa-all-quants premise**: opencoti's CUDA build passes `--fa-all-quants`
  (`build-pipeline.ts` → `-DGGML_CUDA_FA_ALL_QUANTS`), so a native FA-VEC instance for
  every scalar pair already EXISTS. #620 is an **MMA fast-path OVER** that VEC path, NOT
  new support. The env `OPENCOTI_QUANT_MMA_DECODE` (default-on) selects it; env=0 reverts
  each pair to its native FA-VEC path (graceful fall-through, never a boot refusal). Scalar
  KV is not FWHT-rotated, so there is no `wht_o` / de-rotate (unlike turbo #630).
- **Matrix** (D128 served pairs, mirrors the #644 scalar set): hi-K/cheap-V
  {q8K/{q4,q5,q6,q8}V, q6K/q4V} + symmetric diagonal {q4_0, q4_1, q5_0, q5_1, q6_0}.
- **Mechanism** (all additive; nested llama.cpp only): `fattn.cu` — 10 forward-decls +
  `octi_quant_mma_decode_enabled()` (env gate) + the shared predicate
  `octi_scalar_mma_pair_d128(K,V)` (the served D128 scalar pair set); an MMA-dispatch block
  (`OCTI_SCALAR_MMA_D128` macro over the 10 pairs) in
  `ggml_cuda_flash_attn_ext_mma_f16`; guard admittance + a VEC-selector escape in
  `ggml_cuda_get_best_fattn_kernel`, both gated on `dst->src[7]==nullptr` (DCA off), so an
  admitted pair falls straight to `BEST_FATTN_KERNEL_MMA_F16`. 10 new
  `template-instances/fattn-mma-f16-scalar-<K>-<V>-d128.cu` typed instances, each with an
  entry + ncols1/ncols2 gqa dispatch calling
  `ggml_cuda_flash_attn_ext_mma_f16_case<128,128,…,GGML_TYPE_<K>,GGML_TYPE_<V>>`.
  Auto-collected by `build-functions.sh`'s `fattn-mma-*.cu` glob (#531).
- **VALIDATED** (bs2 GPU1, Qwen2.5-14B-1M Q8_0, ctx 65536, n_kv≈44k, correctness via
  logit-equiv `real_frac`, never greedy needle — bug-263/270): beachhead q8_0-K/q4_0-V
  **70.45 tps vs f16 60.99 (1.15×) vs native VEC 30.60 (2.30× — the §3b tax fully killed)**,
  `real_frac 0.0` vs BOTH f16 and native VEC (top-token identical, lossless). Generalization
  (5-config @ ctx 65536): decode tps rises monotonically as V gets cheaper — **f16 62.15 <
  q8q4 68.47 < q8q8 69.45 < q6q4 70.31 < q4q4 73.97**, all boot coherent, `real_frac 0.0` vs
  f16. Snapshot-diff capture (**11 files**: 1 mod `fattn.cu` + 10 new TUs) from a
  reconstructed post-0125 baseline (the pre-#620 `fattn.cu`, verified 4 hunks all
  `octi_scalar_mma`); **byte-identical-verified** (`git apply` reverse-check on live PASS +
  forward `base+0126` == live, all 11 files `cmp`-identical). Marker: `opencoti-hook: P0
  DCA-off scalar-quant MMA decode` (**3 sites** in `fattn.cu`: the predicate + MMA-dispatch
  block; the guard admittance; the selector escape). CUDA-only (host binary unchanged).
  Provenance: opencoti-original. No TS adapter (kernel-routing env, default-on; no
  user-facing server flag — same as `0119`). See
  [evaluations/hal_kv_compression.md](../evaluations/hal_kv_compression.md). **Applies after
  `0125`.**

Chain state after 0126: 72 patches.

### `0127-p0-dcaoff-scalar-mma-decode-d256-d512` (#582-P0 / #674)

**DCA-off scalar-quant MMA decode — D256/D512 generalization.** `0126` (#620) gave a
scalar-quant K/V cache the tensor-core MMA decode fast-path on **D128** (Qwen
full-attention). This patch generalizes that same fast-path to **Gemma-4's two head-dims**:
**D256** (SWA-local layers) and **D512** (global layers — the KV that grows with context,
where the HAL §3c/§3d −28…−35 % kernel-bound quant tax bit hardest). A single Gemma boot now
routes a scalar-KV decode through the tensor cores on **every** layer via the per-layer
head-dim predicate. Making Gemma scalar decode bandwidth-bound (so it beats f16) is what makes
scalar-quant KV the **preferred high-VRAM serving path over turbo** — turbo is slower *and*
decisively lower quality.

- **Mechanism** (0126's, widened; all additive, nested llama.cpp only): the shared predicate
  `octi_scalar_mma_pair_d128` → **`octi_scalar_mma_pair`** now admits symmetric head-dim
  `K==V ∈ {128,256,512}` (was 128-only); the forward-decls become an
  `OCTI_DECL_SCALAR_MMA(kk,vv)` macro emitting all three head-dims; the `OCTI_SCALAR_MMA_D128`
  dispatch macro → **`OCTI_SCALAR_MMA(kk,vv,base)`** routing on `d ∈ {128,256,512}` to
  `base##_d{128,256,512}`. The two consult-sites (the `--fa-all-quants` dead-code guard
  admittance + the selector escape, both `dst->src[7]==nullptr`) already read only that
  predicate, so they generalize for free. **20 new**
  `template-instances/fattn-mma-f16-scalar-<K>-<V>-d{256,512}.cu` (D256/D512 twins of the
  0126 D128 TUs; `ggml_cuda_flash_attn_ext_mma_f16_case<256,256,…>` / `<512,512,…>`). D256/D512
  MMA + in-register scalar V-readers (`ne==8`) are proven by the pre-existing turbo D256/D512
  instances (#630) and f16's `switch_ncols2<256,256>`/`<512,512>`. Auto-globbed (#531).
- **VALIDATED** (bs2 GPU1, Gemma-4 A4B native google 26B-A4B-128e Q4_K_M, **DCA-off**,
  correctness via logit-equiv `real_frac` vs f16 — Gemma A4B PPL/KLD is garbage, so
  correctness is top-token identity; never greedy needle):
  - **ctx 64k** (nkv 42.5k): f16 162.13 < q4q4 170.17 < q8q8 173.86 < **q8q4 174.94** tps
    (q8q4 **+7.9 %** over f16; env=0 VEC fallback **122.17** = the §3c kernel-bound tax → MMA
    is **+43 %** over VEC, tax fully erased). `real_frac 0.0` for q8q4/q8q8/q4q4 (lossless).
  - **ctx 128k** (nkv 112k): f16 152.15 < q4q4 152.95 < **q8q4 154.46** tps (q8q4 **+1.5 %**;
    §3c had q8q4-VEC at **−35 %** here). `real_frac 0.0` for q8q4/q4q4 (lossless at depth).
    Win narrows deep as MoE-FFN + per-step launch overhead dilute the KV-bandwidth delta, but
    scalar stays ≥ f16 and top-token-lossless at both depths.
- Snapshot-diff capture (**21 files**: 1 mod `fattn.cu` + 20 new TUs) from the 0126-level
  `fattn.cu` extracted byte-exact from the `20260714-042251-pre-0126-620-mma-capture`
  vendor-backup tarball (verified 19 D128 refs; diff-vs-live = exactly the #674 generalize);
  **byte-identical-verified** (`base+0127` == live, all 21 files `cmp`-identical). Marker:
  `opencoti-hook: #582-P0 DCA-off quant-MMA decode #620 D128 + #674 D256/D512`. CUDA-only.
  Provenance: opencoti-original. No TS adapter (kernel-routing env, default-on; same as
  `0119`/`0126`). See
  [evaluations/hal_kv_compression.md](../evaluations/hal_kv_compression.md). **Applies after
  `0126`.**
- **Build note**: the 20 new (esp. D512) instances grow `ggml-cuda.so` past 4 GiB (~4.43 GiB),
  which tripped a latent build-pipeline bug (**bug-2178**): `sha256OfFile` read the whole DSO
  into one `ArrayBuffer` and aborted (V8 max typed-array length), skipping the side-load
  mirror. Fixed separately in `packages/opencoti-llamafile/src/build-pipeline.ts` (stream the
  hash — O(1) memory). The DSO link itself holds under the #629 `-mcmodel=large` mitigations.

Chain state after 0127: 73 patches.

### 2026-07-14 — 0128 MTP spec-cycle host parity (bug-858 / #675)

- `// opencoti-hook: mtp-verify-mmvf (bug-858/#675)` (`ggml-cuda/mmvf.cu`, 2 sites) —
  TinyBLAS-gated `src0_small && ne11 <= 8` lift in the **f16 + bf16** ampere_mma branches of
  `ggml_cuda_should_use_mmvf` (siblings of `0093`'s f32 site, bug-2103/#609). Upstream caps
  these at `ne11==1` because its cuBLAS fallback wins past that; TinyBLAS builds have no
  cuBLAS, so MTP verify batches (ne11 = n_accepted+1 = 2..4) dropped the thin bf16
  shared-expert gate GEMVs (`ffn_gate_inp_shexp`) onto a single-block tinyblasGE cliff.
  Base decode (ne11==1) and cuBLAS builds unchanged. Patch `0128`.
- `// opencoti-hook: spec-clone-cheap (bug-858/#675)` (`common/sampling.cpp`) —
  `common_sampler_clone` clones with empty `cur`/`cur_p` instead of deep-copying the
  n_vocab-sized (~1.8 MiB) scratch every speculative cycle: under the cosmo dlmalloc runtime
  that copy is a fresh mmap + fault + munmap = 1.071 ms/cycle (the entire measured spec
  wall-clock gap vs b9859, whose glibc reuses the block). `cur` is write-only scratch
  (`set_logits()` repopulates before any read); the copied `cur_p.data` aliased the source
  sampler's buffer anyway. Output bit-identical on greedy; clone 1.071 → 0.005 ms/cycle.
  **Sync note**: on future bumps, re-check that upstream hasn't started reading a clone's
  `cur`/`cur_p` before `set_logits()` — if it has, restore the copy (and pool it instead).
  Patch `0128`.
- Gate: Qwen3.6-35B NextN n3 282.6 vs b9859 287.4 (−1.7%, was −12%); base parity; accept
  bit-identical. Full matrix + 9-prompt bench:
  [evaluations/three-way-tps.md](../evaluations/three-way-tps.md) "2026-07-14 refresh".

Chain state after 0128: 74 patches.

### 2026-07-14 — 0129 rys-probe nested capture + 0130 parallel≥2 boot/checkpoint robustness (bug-2182/2183)

- `0129-rys-probe-nested-args` — no new markers; completes the `0123`/#669 rys-probe capture:
  the nested `common/common.h` `common_params` fields (`rys_probe*`) and `common/arg.cpp`
  `--rys-probe*` `add_opt` registrations were live in the tree but missing from the chain
  (a fresh apply would fail to compile the outer `rys_probe.cpp`). Chain-integrity only;
  found by the 0130 per-file replay (baseline ≠ live on common.h). The rys hunks sit inside
  regions already marked by `0123`'s rys-probe comments.
- `0130` bug-2182 (`src/llama-kv-cache.cpp`) — static `tie_scalar_f32()` helper + all 20
  dep-tie accumulator sites in the opencoti cpy_k/cpy_v scatter (M2 per-stream, hybrid
  order-agnostic acc2, append-only windowed accumulate/emit) now collapse every tie operand
  to exactly one F32 element (view ONE FULL BLOCK → cast F32 → view 1 element). Previously
  the operand ne was the SOURCE blck_size (32 for q8_0/turbo) → `add([1],[32])` failed
  `ggml_can_repeat` at `-fit`/graph-build under quant KV × parallel≥2 non-unified. All
  inside opencoti-owned scatter code (no new upstream-facing marker; the enclosing blocks
  carry the existing per-stream/rolling-KV hooks). f16/f32 KV graphs byte-identical.
- `0130` bug-2183 (`common/common.h`, `common/common.cpp`,
  `tools/server/server-context.cpp`) — `common_prompt_checkpoint::load_tgt/load_dft`
  return `bool` instead of GGML_ABORTing when `llama_state_seq_set_data_ext` returns 0
  (legitimate under `--kv-unified --parallel≥2` with another slot mid-decode); the server
  task-launch restore site wipes the seq (tgt+dft) and falls back to full re-prefill via
  the existing `do_reset` path; the speculative rollback/draft sites keep fail-fast aborts.
  Comment-tagged `opencoti bug-2183` at all sites. **Sync note**: upstream declares these
  loaders `void` — on future bumps re-apply the bool signature + the server fallback
  branch together (they are one unit; a void loader with the fallback branch dead-codes
  the recovery, an aborting loader with the branch reintroduces the crash).
- Gate: 3090, exact bs2 crash configs — q8 KV × `--parallel 2` non-unified × `-fit on`
  boots n_slots=2 + decodes; forced cell-allocation failure under kv-unified parallel-2 +
  prompt-cache → 0 aborts, FAILED→re-process fallback served all requests.

Chain state after 0130: 76 patches.

## 2026-07-15 — 0131-0133 introspection + version string + chain repair (c3)

### chain repair (bug-2188)
A faithful full replay (clean 0.10.3 archive + Mozilla `llama.cpp.patches` overlay +
strict `git apply` 0001..0130) broke at `0124`: its second hunk duplicate-added the
Mozilla CPU auto-FA block already present from the overlay (the original capture used
an ad-hoc "reconstructed 0123 baseline" that lacked it). The redundant hunk was
dropped from `0124` (functional RYS guard hunk untouched). The same replay also
surfaced `llamafile/rys_probe.cpp` v2 (#673) shipped live but never captured — now
`0131`. **Whole chain 0001..0133 strict-applies from clean 0.10.3 and reproduces the
live tree byte-identically (DIFF_LINES=0).** Rule (bug-2108/bug-2188): capture
baselines come from `.opencoti/s5-build-0078-replay.sh`, never ad-hoc reconstructions.

### `0131-rys-probe-task-battery-v2` (#673)
`llamafile/rys_probe.cpp` — Δtask-ranked balanced battery (see patches README row).
opencoti-owned file (introduced by 0123); no new upstream hook.

### `0132-runtime-introspection` (#676/#677)
Markers: `// opencoti #676` / `// opencoti #677` comments at every touched site.
- `llama.cpp/include/llama.h` — `struct llama_opencoti_kv_info` + `llama_memory_opencoti_kv_info()` (additive API, after `llama_memory_can_shift`).
- `llama.cpp/src/llama-memory.h` — default-false virtual `opencoti_kv_info()` on `llama_memory_i`.
- `llama.cpp/src/llama-kv-cache.{h,cpp}` — impl reading live tensor types + `window_cells` residency.
- `llama.cpp/src/llama-kv-cache-iswa.{h,cpp}` — forward to non-SWA base, `is_iswa=true`.
- `llama.cpp/src/llama-memory-hybrid.{h,cpp}` — forward to `mem_attn`.
- `llama.cpp/src/llama-context.cpp` — C wrapper.
- `llama.cpp/tools/server/server-context.h` — `server_context_meta.json_opencoti` (sleep-safe /props).
- `llama.cpp/tools/server/server-context.cpp` — `get_opencoti_props()`, `get_meta()` init, `/props` key, `slot::to_json` opencoti block, lifetime draft counters (bug-2187).
- `llamafile/BUILD.mk` — bug-2186: `server.cpp.o` rule now depends on `tools/server/*.h` (stale-object ABI-skew guard).
Sync note: on a future llamafile bump, re-anchor the llama.h API block and the
server-context meta field/aggregate-init position; the virtuals are additive.

### `0133-opencoti-version-string` (#613 / c3)
- `llamafile/version.h` — `OPENCOTI_TAG` default + `OPENCOTI_VERSION_STRING` (bump the default together with `vendors/llamafile/OPENCOTI_TAG` on every packaged cut).
- `llamafile/main.cpp` — `--version` + help banner print `llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>)`; `llamafile vX.Y.Z` token must stay FIRST (scripts/llamafile-version.sh contract).
- `llamafile/BUILD.mk` — `OPENCOTI_TAG_FLAG` injection (make var from build pipeline) + `version.h` dep on `main.o`.