docs/protocols/UPSTREAM_SYNC.md

Protocol — Upstream Sync

opencoti is a soft fork of opencode. We sync from upstream regularly. This document defines the rules that keep syncs cheap.

Upstream

• Tracked upstream: mann1x/opencode (dev fork; opencode-of-record is anomalyco/opencode).
• Default upstream branch: dev.
• Sync cadence: weekly is the target, more often when upstream is hot.

The rule

Every opencoti change is either an additive file outside the upstream tree, or a surgical hook inside it.

Additive (preferred)

A file that doesn't exist in upstream opencode. New files always live in:

• docs/
• vendors/sources/<tool>/, vendors/patches/<tool>/
• packages/opencoti-*/ (new bun workspace packages)
• opencoti/ (top-level non-bun roots, e.g. Go for opencoti-server)
• Top-level identity files we own: README.md, CONTRIBUTING.md, SECURITY.md, CLAUDE.md, STATE_SUMMARY.md.

Additive files never conflict on upstream merges.

Surgical hook

A small edit to an existing upstream file. Allowed only when there is no additive way to achieve the effect.

A surgical hook MUST:

1. Be the minimum-line edit that achieves the effect — one import plus one call site, or one conditional.

2. Carry a marker comment on the same line or immediately above:

   // opencoti-hook: <feature-id> — see docs/features/<plan>.md
   

3. Defer to opencoti only when opencoti is enabled. With opencoti disabled (or absent), the upstream behavior must be byte-identical.

4. Be listed in the Registry below.

5. Have an entry in the relevant child plan's "Milestones" or "Surgical hooks" section.

What is NOT a surgical hook

• A refactor of an upstream file "while we're here". File it upstream instead.
• A rename of an upstream symbol. Wrap, don't rename.
• A behavior change that is unconditional. Gate it behind an opencoti flag.

Sync workflow

1. git fetch upstream
2. git merge upstream/dev (or rebase if the branch is local-only).
3. Resolve conflicts:
   • In additive files: there should be no conflicts. If there are, it means the file's path collided with new upstream content — rename our file.
   • In hook sites: keep the hook, accept upstream changes around it. The marker comment is the anchor.
4. Run the conformance check: bun run scripts/check-hooks.ts (to be written — M0) It greps for opencoti-hook: markers and verifies they match the Registry below.
5. Run typecheck and the test suites of every packages/opencoti-* package.
6. Sync STATE_SUMMARY.md (note the upstream SHA we synced to).

Registry — known surgical hooks

Add an entry in the same PR you add the hook.

Hook ID	File	Feature	Plan
caliber-runtime	.gitignore (trailing block)	Caliber pre-commit / session hooks runtime cache	This file (tooling, no per-feature plan)
tiered-inference (import)	packages/opencode/src/session/llm.ts (import block, just below LLMRequestPrep import)	F1 — Tiered Inference Engine	features/tiered_inference.md
tiered-inference (call site)	packages/opencode/src/session/llm.ts (inside run Effect.fn, just before the flags.experimentalNativeLlm seam)	F1 — Tiered Inference Engine	features/tiered_inference.md
tiered-inference (dep)	packages/opencode/package.json (dependencies."@opencoti/tiers": "workspace:*")	F1 — Tiered Inference Engine	features/tiered_inference.md
memory-tools-resolvability (dep)	packages/opencode/package.json (dependencies."@opencoti/memory": "workspace:*")	F4 M5 — Memory / Embedder	features/memory_embedder.md
opencoti-default-plugins (import)	packages/opencode/src/config/config.ts (just below the existing ConfigPlugin import)	F4 M5 follow-up — auto-wire opencoti plugins	features/memory_embedder.md
opencoti-default-plugins (call site)	packages/opencode/src/config/config.ts (in the config-loader Effect.fnUntraced body, just before the return { config: result, … })	F4 M5 follow-up — auto-wire opencoti plugins	features/memory_embedder.md
tiered-inference-setup (import)	packages/opencode/src/index.ts (just after the existing isRecord import)	F1 M5.7 — Tier-0 setup CLI	features/tiered_inference.md
tiered-inference-setup (call site)	packages/opencode/src/index.ts (.command(tierCommand) after .command(DbCommand))	F1 M5.7 — Tier-0 setup CLI	features/tiered_inference.md
opencoti-server-autostart (import)	packages/opencode/src/config/config.ts (just below the opencoti-default-plugins import)	F3 M3 — opencoti-server TS client + autostart	features/opencoti_server.md
opencoti-server-autostart (schema)	packages/opencode/src/config/config.ts (opencoti: Schema.optional(Schema.Struct({ server: ... })) block in the main config schema)	F3 M3 — opencoti-server TS client + autostart	features/opencoti_server.md
opencoti-server-autostart (call site)	packages/opencode/src/config/config.ts (in the config-loader Effect.fnUntraced body, just after the opencoti-default-plugins call site, before return { config: result, … })	F3 M3 — opencoti-server TS client + autostart	features/opencoti_server.md
opencoti-server-client (dep)	packages/opencode/package.json (dependencies."@opencoti/opencoti-server-client": "workspace:*")	F3 M3 — opencoti-server TS client + autostart	features/opencoti_server.md

The two TS edits carry // opencoti-hook: tiered-inference — see docs/features/tiered_inference.md markers; the package.json dep has no anchor comment (JSON has no comments) — this registry table is its only anchor. On upstream merges, if upstream renames any of these files or restructures the run Effect.fn around the call-site marker, the marker comments remain the canonical pointer for re-applying the hook.

F3 M4 (2026-05-23): no new registry rows. M4 (session + turn events flowing into opencoti-server) ships as a pure plugin (@opencoti/session-events-plugin) that subscribes to opencode's typed Hooks.event firehose. Since the plugin API already exposes session.created/updated/deleted/error/idle as typed events, no surgical hook into upstream source is required — matching the M5-D2 precedent. Hook count stays at 9 source markers + 4 JSON deps from F3 M3. See docs/features/opencoti_server.md "M4" for the design-pivot rationale.

F3 M5 (2026-05-23): no new registry rows. M5 (tier-engine event sink) ships as a plugin (@opencoti/tier-events-plugin) that injects a Telemetry implementation into @opencoti/tiers via a new module-level slot (registered-telemetry.ts, mirroring the existing active-config.ts overlay pattern). The slot is consulted by runtime.ts:99 after the explicit hook arg and test-injected exec opts, before noopTelemetry — adding the sink without touching upstream source. The Telemetry interface itself was added during F1 M5 with the docstring already saying F3 would implement it, so M5 lands in a seam that was waiting for it. Hook count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4 and M5). See docs/features/opencoti_server.md "M5" for the seam design.

F3 M6 (2026-05-23): no new registry rows. M6 (daemon tool registry → opencoti sessions) ships as a plugin (@opencoti/server-tools-plugin) that fetches the daemon's /v1/tools catalog and injects a Record<string, Tool> into @opencoti/tiers via a new module-level slot (registered-server-tools.ts, mirroring M5's registered-telemetry.ts). The slot is consulted by runtime.ts's mergeTools call last — after the synthetic-tier and memory-bridge tool records that were already merged there since F1/F4 M5 — so the daemon tools land in a seam that already existed for exactly this kind of record-merge, no upstream source touched. Hook count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4, M5, and M6). See docs/features/opencoti_server.md "M6" for the design-pivot rationale.

F3 M7 (2026-05-23): no new registry rows. M7 (the pgvector storage backend) is entirely additive Go inside the opencoti/server/ root — a new internal/store/pgvector/ package implementing the same Store interface family as internal/store/sqlitevec/, plus a shared internal/store/storetest/ conformance harness and a pure-logic internal/store/common.go both backends share. Backend selection is a serve-time CLI flag (--backend sqlite|pgvector), not a build flag, so no upstream file and no TS package is touched. Unlike M4–M6, M7 ships no plugin — there is no opencode-side surface to wire, since the backend is purely internal to the daemon. Hook count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4, M5, M6, and M7). Note: M7 also moved the module + host Go toolchain to 1.25 for the modern pgx/pgvector-go/testcontainers releases — a build-environment change, not a sync-surface change. See docs/features/opencoti_server.md "M7" for the backend design.

F3 M8 (2026-05-23): no new registry rows. M8 (multi-session coordination — presence, broadcasts, advisory locks) is additive Go inside opencoti/server/: a new in-memory internal/coord hub plus an internal/server/coord.go HTTP surface (including the daemon's first SSE endpoint), injected as a concrete dependency on server.Options. The new @opencoti/coordination-plugin is the ecosystem's first SSE consumer, but it wires in like M4–M6: a string appended to OPENCOTI_DEFAULT_PLUGINS in opencoti-owned @opencoti/tiers, consumed by the already-registered opencoti-default-plugins call site — not a new source marker. The count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4–M8). No upstream file and no store backend touched. See docs/features/opencoti_server.md "M8".

F3 M9 (2026-05-23): no new registry rows. M9 (opt-in shared context) is additive Go inside opencoti/server/: a new internal/share package (the share coordinator) plus an internal/server/share.go HTTP surface, both reusing the existing Store ACL methods (SetSessionACL / ListCollections) and the M8 coord.Hub — no new Store method, no schema change, no store backend touched. The @opencoti/coordination-plugin is extended in place to consume the two new collection.shared / collection.unshared bus events and to add an opt-in share_session_collection flag; it remains auto-wired through the already-registered opencoti-default-plugins call site, so no new source marker. The count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4–M9). See docs/features/opencoti_server.md "M9".

F3 M10 (2026-05-24): no new registry rows. M10 (the sqlite↔pgvector migrate subcommand) is additive Go inside opencoti/server/: a new MigrationStore optional interface on internal/store plus a per-backend migrate.go implementing it and a new cmd/opencoti-server/migrate.go subcommand. Like M7 it has no opencode-side surface — no plugin, no HTTP endpoint, no TS client method — so there is nothing to auto-wire and no source marker to add. It touches no upstream file, no schema (dedupe reuses existing columns/indexes), and no go.mod/go.sum (deps already present; the new code is stdlib encoding/binary + already-vendored drivers). The count stays at 9 source markers + 4 JSON deps from F3 M3 (unchanged through M4–M10). See docs/features/opencoti_server.md "M10".

F5 M0–M3 (2026-05-24 → 2026-05-29): no new opencode source markers. The Advanced KV Cache series (0006–0030) lives entirely in the vendored vendors/sources/llamafile/ tree, captured into numbered patches under vendors/patches/llamafile/. The 6 surgical hooks added during F4 M5+M3 (tiered-inference x3, memory-tools-resolvability, opencoti-default-plugins x2 — see rows above) stay unchanged. F5 M3 specifically also adds two NEW non-upstream files inside the vendored tree: vendors/sources/llamafile/llama.cpp/ggml/include/ggml-neo-pipeline.h and .../ggml/src/ggml-neo-pipeline.cpp (the NEO orchestrator). These do not belong in this registry — they are not surgical hooks into upstream opencode, they are additions to vendored llama.cpp (the distinction in "What is NOT a surgical hook" above). They are tracked in vendors/patches/llamafile/README.md's 0030 row, which is the canonical vendored-source registry.

F5-opt W1–W2 (2026-05-30): still no new opencode source markers. The concurrency=1 optimization round (0036, 0040) likewise lives entirely in vendored llama.cpp. 0036 (W1, PCIe probe) adds NEW non-upstream common/pcie-profile.{h,cpp}. 0040 (W2, wholesale iqk CPU-FA import) adds the NEW vendored ggml/src/iqk/** engine tree (29 files), plus ggml/include/ggml-iqk-flash-attn.h + ggml/src/ggml-iqk-flash-attn.cpp, and carries one opencoti-hook: f5-opt-cpufa marker in the vendored ggml/src/ggml-cpu/ops.cpp dispatch site — which, being under vendors/sources/, is not counted by the packages/ docs/ grep and is registered solely via the 0040 README row. The opencoti-llamafile adapter wires --iqk-flash-attn through plain config (no marker). Marker count in packages/ docs/ is therefore unchanged: 20 per git grep "opencoti-hook:" packages/ docs/ | wc -l (this counter has read 20 since F4 M5+M3; earlier "18/19" figures in this file are stale prose, not a real source delta). See docs/features/advanced_kv.md for the per-milestone history.

F5-opt W3–W4 + F5 M6 PolyKV/MTP (2026-05-30 → 2026-06-07): still no new opencode source markers. 0042 (W3, fused MoE up+gate), 0070 (W4, M7 Rolling KV), 0071 (cosmocc state-IO, bug-269), 0072/0073 (M6 PolyKV — SharedKVPool + TurboQuant), and 0074 (M6-S4 — speculative MTP draft head, #373/#404-410) all live entirely in vendored llama.cpp (plus, for 0071, the outer-vendor llamafile/llamafile.{c,h}, and for 0073, the outer-vendor llamafile/build-functions.sh) — none are hooks into upstream opencode source, so none are counted by the packages/ docs/ grep; each is registered solely via its README.md row. 0074 touches only nested llama.cpp/ files (24 files, 1 new: src/models/gemma4-assistant.cpp), no outer-vendor delta; its three surgical upstream edits + the additive surface are detailed in the F5 M6-S4 block below.

Update 2026-06-16 (llamafile 0.10.3 bump): 0074 — and the gemma-specific 0076 / 0077 — are now RETIRED, superseded by upstream-native NextN/MTP + gemma4 E-series support in dbe9c0c (the +613-commit jump under the 0.10.3 bump). See the per-patch RETIRED banners below and docs/protocols/LLAMAFILE_UPGRADE.md. The active stack is 35 patches (0001-0007, 0010, 0020, 0030-0036, 0040, 0042, 0070-0073, 0075, 0078, 0079, 0080, 0081, 0082, 0083, 0084, 0085, 0086, 0087, 0088, 0089); the three retired entries are kept below for audit history only (their patch files no longer exist). Re: 0079 — the gemma assistant drafter (retired 0074) is restored on 0.10.3 as 0079-gemma-assistant-mtp.patch to coexist with upstream-native NextN as --spec-type draft-assistant (#423, a deliberate two-engine union): native NextN does not cover Gemma-4 (iSWA, no NextN head), so the assistant drafter is Gemma's only MTP path. See the opencoti F5 M6-S4 mtp (0.10.3 restore) block near the end of this file.

0070 marker count (S4 update, 2026-06-04): the R2-a-era "6 markers" figure is superseded. After the full Stage-3 position-window forward + the Stage-4 --kv-residency-mode knob landed (and S4-2 removed the throwaway iSWA-spike

• 29 env-gated diagnostic scaffolds), 0070 now carries **40** opencoti-hook: f5-rolling-kv / rolling-kv M7 markers across ggml/include/{ggml.h,ggml-iqk-flash-attn.h}, ggml/src/{ggml-backend.cpp}, ggml/src/ggml-cpu/{ggml-cpu.c,ggml-cpu.cpp,ops.cpp,ops.h}, ggml/src/ggml-cuda/{fattn.cu,fattn.cuh,fattn-common.cuh,ggml-cuda.cu}, ggml/src/iqk/iqk_flash_attn.cpp, src/{llama-context.cpp,llama-graph.cpp} (the canonical inventory is git grep -n "opencoti-hook:" vendors/sources/llamafile/llama.cpp = 54 total incl. the f5-opt-fmoe/f5-opt-cpufa markers from 0040/0042). The exact set is descriptive, not load-bearing: the 0070 README row is the registry of record. 0071 carries no new in-tree marker — its 3-file edit (the llamafile_open_plain def/decl + the llama-mmap.cpp COSMOCC fallback) is registered via the 0071 README row. Counter correction: the canonical opencode-source-hook figure is git grep -h "opencoti-hook:" -- packages/ = 10, unchanged by W3/W4. The combined packages/ docs/ counter now reads 22 (the W1–W2 note above cites 20); that +2 is pure docs prose — this file's W2 paragraph and advanced_kv.md both now mention f5-opt-cpufa by name — not new source hooks. That combined counter has always conflated real markers with documentation mentions; treat packages/-only = 10 as canonical, and the vendored-patch markers as registered by their README.md rows.

F5 M6 PolyKV S1 (2026-06-05): no new opencode source markers. The opencoti-hook: poly-kv-pool markers (git grep -n "poly-kv-pool" vendors/sources/llamafile/llama.cpp/tools/server = 4: 1 param block in server-task.h, 2 parse lines in server-task.cpp, 1 gated prefill branch in server-context.cpp) live entirely in vendored llama.cpp server source and are additive + shared_pool_slot < 0-default-off (boot byte-identical). Captured as vendors/patches/llamafile/0072-poly-kv-pool.patch at M6-S3 (#372) and registered via that patch's README.md row — same mechanism as 0070/0071. The packages/-only canonical counter stays 10. S1 runtime prerequisite: the server must boot --no-clear-idle so the read-only pool slot's KV stays device-resident for cross-seq seq_cp sharing (see poly_kv.md S1 / [[bug-367]]).

F5 M6 PolyKV S2 perf-path Level A (2026-06-06): GPU-resident turbo dequant-on-lift. New opencoti-hook: turboquant perf-lift markers — git grep -n "turboquant perf-lift" vendors/sources/llamafile/llama.cpp = 10 across 6 files: 3 in new additive opencoti files (ggml/src/ggml-cuda/turbo-cpy.cu, turbo-cpy.cuh, turbo-innerq-dev.cuh — file headers) and 7 in 4 edited vendored files (ggml-cuda/set-rows.cu ×2 = include + accessor impl; ggml-cuda/cpy.cu ×3 = 2 includes + 1 dispatch branch; ggml-cuda/ggml-cuda.cu ×1 = CPY supports_op accepting turbo{2,3,4,8}_0→F16; src/llama-graph.cpp ×1 = single-cast turbo→F16 lift). All additive + default-inert (the CPY branch only fires for a turbo→f16 cast, which only exists when -ctk/-ctv turboN is set; f16/q8/q4 paths untouched, boot byte-identical). Captured in vendors/patches/llamafile/0073-turboquant-kv.patch at M6-S3 (#372) and registered via that patch's README.md row — same mechanism as 0070/poly-kv-pool. The packages/-only canonical counter stays 10. See poly_kv.md S2 perf-path Level A.

F5 M6 PolyKV S2 perf-path Level B (2026-06-06): fused FA-VEC turbo + new GGML_OP_TURBO_WHT op. New opencoti-hook: turboquant perf-lift (… Level B) markers, additive on top of Level A. New ggml op GGML_OP_TURBO_WHT appended at the end of the op enum (after FLASH_ATTN_TAIL_PARTIAL, before GGML_OP_COUNT) — same append-at-end discipline as MOE_FUSED_UP_GATE/STREAMING_FLASH_ATTN; the host/DSO GGML_OP_COUNT (now 100) and the name/symbol arrays + both static_asserts in ggml.c move in lockstep, CPU forward is an abort-stub (CUDA-only op, like STREAMING_FLASH_ATTN). New additive files: ggml-cuda/turbo-wht.{cu,cuh} + turbo-wht-core.cuh, ggml-cuda/fattn-turbo.cuh, template-instances/fattn-vec-instance-turbo{2,3,4}_0-*.cu (3, generator-emitted). Edited vendored files (surgical, default-inert — only fire when -ctk/-ctv turbo{2,3,4} is set): ggml.h+ggml.c (op enum + ggml_turbo_wht ctor), ggml-cpu/ggml-cpu.c (abort case + planner n_tasks=1), ggml-cuda/ggml-cuda.cu (TURBO_WHT dispatch + supports_op), ggml-cuda/fattn-common.cuh (turbo get_vec_dot_KQ/get_dequantize_V cases), ggml-cuda/fattn-vec.cuh (is_turbo_K config), ggml-cuda/fattn.cu (turbo→VEC gate), template-instances/generate_cu_files.py (turbo instance loop), src/llama-graph.cpp (build_attn_mha fused gate + prefill-hybrid TBV_FUSE_MAX_NQ). f16/q8/q4 paths untouched, boot byte-identical. Captured together with Level A in 0073-turboquant-kv at M6-S3 (#372). See poly_kv.md S2 perf-path Level B.

F5 M6 PolyKV patch numbering (S3, #372): 0072/0073, NOT the plans' stale 0060/0061. The build pipeline applies patches in pure lexical sort order (build-pipeline.ts, patches.sort()); there is no non-lexical override. The TurboQuant FA-VEC edits (fattn.cu, llama-graph.cpp, ggml-cuda.cu) sit on top of 0070's rewrite of those same files, so they MUST sort after 0071 — 0060/0061 would sort before 0070 and break the chain (M7 shipped before M6, so the original plan labels predate it). 0073 also carries the outer-vendor llamafile/build-functions.sh host-build source-list add for the new ggml-turbo-quant.c TU. Both patches reverse-apply clean against live and forward-reproduce the live tree byte-for-byte (snapshot-diff capture, same as 0070/0071). S0 asymmetric -ctk q8_0 -ctv q4_0 needs no patch (rides M7 dequant-on-lift natively).

F5 M6-S4 P2 (#406, 2026-06-07): Gemma4 MTP speculative-decode graph + inert (verify-only) scheduler. Ported from the AtomicBot atomic-llama-cpp-turboquant fork. All new code carries the // opencoti F5 M6-S4 mtp marker (same marker family as P1). Default-inert: nothing in the normal decode/server path invokes the MTP machinery — llama_decode_mtp is the only entry and it is called by nobody in-tree at P2 (active drafter is P3/P4). The patch is additive-dominant with three surgical edits to existing upstream functions that must be re-applied on upstream sync (the marker comment is the real anchor; line numbers below are best-effort):

• src/llama-graph.h — llm_graph_params::allow_reuse token_compat relaxation (~line 572). The inlined ubatch token/embd compatibility expression is replaced with an MTP-aware named token_compat bool: MTP graphs feed BOTH token and embd, so steps 1..N-1 of one decode_mtp can reuse the encoded graph. Default (non-MTP) branch is byte-identical to upstream.
• src/llama-graph.cpp — llm_graph_result::reset() nulls t_argmax (~line 805). One added line next to the other t_* = nullptr; lines so the new MTP-only t_argmax field is reset like its siblings.
• src/llama-model.cpp — build_graph case LLM_ARCH_GEMMA4 MTP sub-branch (~line 8723). The target arch case gains a params.gtype == LLM_GRAPH_TYPE_MTP branch that builds llm_build_gemma4_mtp from the nested mtp_assistant; the else (iswa) path is unchanged. The LLM_ARCH_GEMMA4_ASSISTANT case keeps its P1 throw-stub.

Everything else is purely additive (new functions/fields/file, no upstream-line mutation, marker-tagged): LLM_GRAPH_TYPE_MTP enum value; llm_graph_input_mtp class (+set_input/can_reuse); t_argmax field + get_argmax() accessor; llm_graph_context::build_attn_mtp; the new file src/models/gemma4-assistant.cpp (+ its source-list entry in both llama.cpp/BUILD.mk — the authoritative cosmocc build list, the one the host binary actually links — and src/CMakeLists.txt as a mirror; + llm_build_gemma4_mtp decl in src/models/models.h); llama_kv_cache::mtp_slot_info + llama_kv_cache_iswa::init_mtp; the synchronous llama_context MTP machinery (graph_params_mtp, ensure_sched_mtp, process_ubatch_mtp, graph_compute_mtp, decode_mtp facade → decode_mtp_sync) + members sched_mtp/gf_res_prev_mtp; the C API llama_decode_mtp (+ include/llama.h decl); and a #include "llama-kv-cache-iswa.h" in llama-context.cpp. Adaptations vs fork (documented at the call sites): build_attn_mtp passes ggml_turbo_wht(..., /*scale=*/nullptr) instead of the absent get_turbo_innerq_scale_inv() (OURS folds the scale into the device-resident kernel); the decode_mtp facade is rerouted unconditionally to decode_mtp_sync (async worker is P4); ensure_sched_mtp/graph_compute_mtp drop the async-only mtp_worker spawn and backend_cfg_mu lock (sync-only P2 path). Known risk R1 (double-Q-rotation): if the MTP cross-read target K/V enter build_attn_mha as turbo (fused branch), Q is rotated twice — graph still builds (P2 inert), correctness is a P3 gate.

⚠️ RETIRED 2026-06-16 — llamafile 0.10.3 bump (#464, B2). 0074-speculative-mtp.patch is dropped from the rebased stack. The nested llama.cpp base advanced 5e9c63546→dbe9c0c (+613 commits; pin vendors/pin/llamafile.txt → 7fca8b2 0.10.3), which ships upstream-native NextN/MTP: 255582687 "llama + spec: MTP Support (#22673)" (NextN self-attention MTP graph + the n_rs_seq recurrent rollback ring, a proven ancestor of dbe9c0c). The hand-wired gemma4_assistant draft head + its tok_embd-on-OUTPUT-buft placement fix are superseded for throughput; qwen MTP rebases onto upstream-native --spec-type draft-mtp, but the gemma assistant drafter — which native NextN does not cover (Gemma-4 is iSWA, no NextN head) — is restored on 0.10.3 as 0079-gemma-assistant-mtp.patch, coexisting as --spec-type draft-assistant (#423). The 0.10.1-base 0074 diff is dropped (it did not apply across the +613-commit jump); the feature lives on in 0079 — see the active block near the end of this file. The block below is retained for audit history; the patch file no longer exists. See docs/protocols/LLAMAFILE_UPGRADE.md.

F5 M6-S4 P3/P4 (#407-410, 2026-06-07): active drafter + throughput win — CAPTURED as 0074-speculative-mtp.patch. P3 activated the synchronous single-token MTP forward (decode_mtp → decode_mtp_sync, process_ubatch_mtp, the C API) and proved it logit-equivalent to plain greedy (.opencoti/s4-p3-logit-equiv.sh: real_frac=0.0, 100% draft acceptance). P4 made it a measured decode-throughput win: the assistant's weight-tied dense LM head (tok_embd, 277 MiB) is classified as an INPUT tensor and was hard-pinned to the CPU buft (`llama-model.cpp:2716 dev_input), so the per-draft-step full-vocab (262144) output mul_mat ran as a **CPU island** that stalled the GPU (~95% idle, GGML_SCHED_DEBUG=2). The fix (one surgical edit, marker-tagged) routes the gemma4_assistant tok_embd through the **OUTPUT** buft (dev_output, GPU when the output layer is offloaded) instead of dev_input→ MTP-on **114.0 tok/s vs off 78.3 = 1.46×** (seq 113.5; seq==fused byte-identical; acceptance 0.769/0.606).0074reverse-applies clean against live and forward-reproduces the live tree byte-for-byte (snapshot-diff capture from base0001-0042 + 0070 + 0071 + 0072 + 0073, same mechanism as 0070-0073; verified both directions, 24 files, 0 diffs). TS adapter knobs: mtpHead/specType/draftNGpuLayers (--mtp-head/--spec-type mtp/-ngld) in packages/opencoti-llamafile/src/{config,launch}.ts. The reverted sparse-argmax centroid fast path (dead for this dense GGUF) is banked at .opencoti/banked/mtp-sparse-argmax.md`, not shipped.

F5 M6-S2 #411 (2026-06-07): D=512 fused FA-VEC for turbo2/turbo3 — CAPTURED as 0075-d512-turbo-vec.patch. 0073's Level-B fused turbo read covered head_dim∈{128,256}; the Gemma-4 A4B GLOBAL layers use head_dim 512 (key_length=512) and therefore fell back to the Level-A turbo→f16 materialize (a decode tax that scales with the global-cache n_kv). 0075 adds the D=512 fused FA-VEC instances for turbo2/turbo3 (DECL_FATTN_VEC_CASE(512, …) in fattn-vec.cuh + the two template-instance .cu + generate_cu_files.py autogen line; fattn.cu kernel-selection relax + dispatch; llama-graph.cpp build_attn_mha fused_turbo_512 gate). The VEC kernel is already D-generic (nthreads/ nthreads_V derive from D arithmetically; at D=512/ncols=1 the KQ smem is ~4–8 KB and VKQ ~16 regs — well within the 48 KB static-smem cap, verified compile + run clean). No new surgical hook — every edit sits inside 0073's existing turbo regions; no new ABI; no new flag (the D=512 fused path auto-engages for turbo2/3 decode via the existing TBV_FUSE_MAX_NQ knob, default 8). Correctness is Parseval-EXACT (the WHT is block-diagonal over 128-blocks, so D=512 = 4 Parseval-exact 128-blocks): fused-vs-materialize logit-equivalence real_frac=0.0 for BOTH turbo3 and turbo2 (.opencoti/p411-d512-gate.sh). Perf — turbo3 @ ctx 27.7k, fused 41.0 vs materialize 29.7 tok/s = +38%, scaling with n_kv (.opencoti/p411-perf.sh). Snapshot-diff capture (6 files, 95 lines, 0 new) from base 0001-0042 + 0070 + 0071 + 0072 + 0073 + 0074; verified both directions, 0 diffs. No TS adapter change (no user-facing flag).

When you add a hook, append a row above, replacing the placeholder with: stable hook id, file path + best-effort line number (line moves on upstream syncs are normal — the marker comment is the real anchor).

Why we're strict

Every undiscoverable edit to an upstream file is a future merge conflict that someone (probably us) will resolve at 2 AM during a release crunch. Strictness up front is the cheapest tax.

opencoti F5 gemma4-shared-kv (F5 — elastic E-series KV-layer-sharing) — ⚠️ RETIRED 2026-06-16

RETIRED at the llamafile 0.10.3 bump (#464, B2). dbe9c0c loads gemma4 gemma4.attention.shared_kv_layers natively (its loader no longer requires attn_k/attn_k_norm on every layer), so the TENSOR_NOT_REQUIRED loader gate described below is superseded by upstream native support and was dropped from the rebased stack. 0076-gemma4-shared-kv.patch no longer exists; entry kept for audit history. See docs/protocols/LLAMAFILE_UPGRADE.md.

One vendored-source marker (// opencoti F5 gemma4-shared-kv (#417)) in vendors/sources/llamafile/llama.cpp/src/llama-model.cpp, in the LLM_ARCH_GEMMA4 create_tensor loop. The elastic Gemma-4 E-series (E2B/E4B) declare gemma4.attention.shared_kv_layers (E2B 20/35, E4B 18/42): the trailing layers carry no attn_k/attn_k_norm and reuse an earlier layer's KV cache. The graph side (src/models/gemma4-iswa.cpp hparams.has_kv(il) guard), hparams.has_kv() (src/llama-hparams.cpp), and the KV-cache layer-reuse callback (llama-model.cpp, the arch == LLM_ARCH_GEMMA4 layer_reuse_cb) were already brought over by the MTP/TurboQuant port; the only missing piece was the loader requiring attn_k/attn_k_norm on every layer. The hook gates both with const int kv_flags = hparams.has_kv(i) ? 0 : TENSOR_NOT_REQUIRED;. Inert for dense Gemma-4 (A4B/12B/31B): they set n_layer_kv_from_start = n_layer, so has_kv(i) is true for all i and kv_flags ≡ 0 — the create_tensor calls are bit-identical to upstream. Captured as 0076-gemma4-shared-kv.patch (snapshot-diff vs the 0075-applied base; reverse-applies clean + forward-reproduces live byte-for-byte). Unblocks E2B/E4B load (was bug-438). Marker convention: this is a vendored-source // opencoti <feature> marker (patches/README §"In-tree markers vs surgical hooks") — NOT an // opencoti-hook: marker, so the git grep "opencoti-hook:" packages/ docs/ inventory is unchanged.

opencoti F5 M6-S4 mtp ordered-gpu (F5 M6-S4 — ordered-embeddings draft-head GPU placement) — ⚠️ RETIRED 2026-06-16

RETIRED at the llamafile 0.10.3 bump (#464, B2). This gemma4-MTP ordered-drafter GPU-placement hook is a follow-on to 0074 and is superseded by the same upstream-native NextN/MTP convergence (255582687, #22673) under dbe9c0c. 0077-gemma4-mtp-ordered-gpu.patch was dropped from the rebased stack; entry kept for audit history. See docs/protocols/LLAMAFILE_UPGRADE.md.

One vendored-source marker (// opencoti F5 M6-S4 mtp ordered-gpu (#418)) in vendors/sources/llamafile/llama.cpp/src/llama-model.cpp, in the LLM_ARCH_GEMMA4_ASSISTANT create_tensor path inside if (hparams.use_ordered_embeddings). The follow-on to 0074's dense tok_embd fix: the ordered ("efficient") draft head reads mtp_centroids (centroid mul_mat) and mtp_token_ordering (get_rows) every draft step, and llama.cpp default-classifies both as inputs pinned to the CPU buft (dev_input) — a CPU island that fragments the MTP graph (GGML_SCHED_DEBUG splits=2, assistant CPU_Mapped 277 MiB) and makes MTP-on slower than off on the ordered drafters (E2B/E4B). The hook routes both through the OUTPUT buft (pimpl->dev_output.buft_list, GPU when the output layer is offloaded) via the ml.create_tensor(hparams, &pimpl->cpu_buft_list, dev_output…) form — identical mechanism to 0074. Result: assistant CPU_Mapped 277→0.80 MiB, draft dur(g) 783→300 ms; E2B 0.87×→1.22×, E4B 1.34×→1.42× (RULER 32k, q8_0 KV; acceptance bit-identical pre/post). Inert for the dense drafters (A4B/12B/31B): they set use_ordered_embeddings=false and never enter this branch. Captured as 0077-gemma4-mtp-ordered-gpu.patch (snapshot-diff vs the 0076-applied base; reverse+forward byte-identical). Vendored-source marker — not counted by the opencoti-hook: inventory.

opencoti F5 dca (F5 — Dual Chunk Attention, training-free long-context)

The DCA marker family // opencoti F5 dca (187 occurrences) spans the vendored llama.cpp tree — this is a substantial additive feature, not a one-line hook. Surface:

• New files vendors/sources/llamafile/llama.cpp/src/dca.{cpp,h} — the three-regime builder build_attn_dca / build_attn_dca_core / build_attn_inp_dca and the ggml_flash_attn_ext_dca_fused host wrapper (INTRA pos%c / SUCC +c / INTER 2c, merged by online-softmax).
• KV-cache fill src/llama-kv-cache.cpp set_input_dca — #635 INTER-Q = 2c dense position-based fill + the #617B sticky is_fragmented fragmented-mask fallback (masks left NULL on the contiguous fast path).
• Per-arch dispatch hooks in src/models/{gemma4-iswa,qwen2,qwen3,qwen35,qwen35moe,qwen3moe}.cpp (+ models.h, llama-arch.cpp, llama-graph.{h,cpp}, llama-kv-cache.h, llama-kv-cells.h, llama-context.cpp) — gemma4 hd512 (global layers), qwen2/3 hd128, qwen3.5 family hd256, incl. the qwen35 nextn_predict_layers MTP-block skip (#622).
• Runtime knobs src/llama-cparams.h (dca_enabled / dca_chunk_size / dca_yarn_factor) + common/{arg,common}.{h,cpp} args --dca / --dca-chunk-size / --dca-yarn-factor (env LLAMA_ARG_DCA*); tools/server/server-context.cpp. --dca-yarn-factor is consumed in build_dca_rope (dca.cpp) as the YaRN mscale — dca_attn_factor = attn_factor * cparams.dca_yarn_factor into ggml_rope_multi/ggml_rope_ext (marker // opencoti-hook: dca-yarn-factor (bug-740 / #550), patch 0087; default 1.0 ⇒ inert/byte-identical). Characterized as no-win for retrieval (over-sharpens; default 1.0 correct — see README 0087 row).
• New ggml op GGML_OP_FLASH_ATTN_EXT_LSE (ggml/include/ggml.h + ggml.c, ggml-backend.cpp, ggml-cpu/*, and the fused MMA kernel + LSE combine in ggml/src/ggml-cuda/{fattn*,ggml-cuda.cu}): flash-attention emitting a NORMALIZED O packed with per-row lse, so the three regimes merge exactly.

Default-OFF (cparams.dca_enabled=false) is byte-identical to pre-0078 (the contiguous fast path leaves the fragmented masks NULL). Constraints today: head_dim in {128,256,512}, DK==DV, f16 KV required (dca.cpp assert — to be generalized to all KV types, incl. bf16 / scalar / turbo / TCQ, in the follow-on all-KV-types milestone). Captured as 0078-dca.patch (clean-room capture from the sibling opencoti-dca repo, 37 files / 2 new; landed byte-identical, git apply --check clean on the 0077-applied tree, production-clean of DCA617* debug). Marker convention: these are vendored-source // opencoti <feature> markers (patches/README §"In-tree markers vs surgical hooks") — there are zero // opencoti-hook: markers in 0078, so the git grep "opencoti-hook:" packages/ docs/ inventory is unchanged. ABI: adds the GGML_OP_FLASH_ATTN_EXT_LSE ggml op — the cosmocc binary + ggml-cuda.so DSO must be rebuilt and paired.

opencoti F5 M6-S4 mtp (0.10.3 restore) (F5 M6-S4 — Gemma-4 assistant-drafter MTP, restored onto 0.10.3) — 0079

The gemma assistant drafter retired as 0074 at the 0.10.3 bump (its 0.10.1-base snapshot-diff did not apply across the +613-commit jump to dbe9c0c) is restored on the new base as 0079-gemma-assistant-mtp.patch (#423), re-exposing it as a first-class --spec-type draft-assistant that coexists with upstream-native NextN MTP (--spec-type draft-mtp). This is a deliberate two-engine union, not a reversal of the retirement: native NextN covers the qwen-family NextN archs; the assistant drafter covers Gemma-4, which has no native NextN head (iSWA) — so for Gemma it is the only MTP path. The user mandate (#423) is explicit: do not drop ours; extend/restore it so it works alongside every other development (PolyKV, rolling-KV, turbo, DCA).

No enum/graph collision: COMMON_SPECULATIVE_TYPE_MTP (ours) vs COMMON_SPECULATIVE_TYPE_DRAFT_MTP (native); LLM_GRAPH_TYPE_MTP (ours) vs LLM_GRAPH_TYPE_DECODER_MTP (native). The ~0.4B Q-only cross-attention drafter (LLM_ARCH_GEMMA4_ASSISTANT) loads into the gemma4 target via llama_model_load_mtp_from_file (no second llama_context, no second KV cache) and reads the target KV read-only at decode (build_attn_mtp). It rides the 0.10.3 seq-aware speculative framework (common_speculative_process/_draft/_accept/_need_embd public hooks, which replace the old server h_idx/seq_id/set_h_idx plumbing) — which is why it composes with PolyKV / multi-seq for free.

All new code carries the // opencoti F5 M6-S4 mtp marker (vendored-source, not opencoti-hook: — registered via the 0079 README row + this block; the packages/ docs/ counter is unchanged). The patch is additive-dominant; the surgical edits to existing 0.10.3 upstream functions that must be re-applied on the next upstream sync (the marker comment is the real anchor; line numbers best-effort):

• src/llama-graph.cpp — llm_graph_input_attn_kv_iswa::set_input buffer-guard decouple (bug-508). 0.10.3 nests the set_input_kq_mask call inside if (self_k_idxs && self_k_idxs->buffer). The read-only MTP draft graph (build_attn_mtp) prunes the KV-write idxs (their galloc buffer is null) yet still reads the mask — so the nested guard would skip the live mask and feed garbage (a silent re-introduction of bug-404). Fix: decouple — each self_{k,v}_idxs / self_kq_mask{,_swa} / rot setter now guards on its own ->buffer. Byte-identical for every non-MTP graph (all buffers live ⇒ every branch taken, as upstream).
• tools/server/server-context.cpp — leading has_dft() && draft-assistant branch. Before the existing has_dft() draft-model path (now else if), a new branch detects COMMON_SPECULATIVE_TYPE_MTP in --spec-type, calls llama_model_load_mtp_from_file(model_tgt, params_dft.model.path, mparams_dft), and leaves model_dft/ctx_dft null (null-safe: every ctx_dft checkpoint path early-returns on a null context).
• per-arch loader + graph-dispatch plumbing — a per-model class llama_model_gemma4_assistant registered via llama_model_mapping (the 0.10.1 giant-switch loader is gone) and a params.gtype == LLM_GRAPH_TYPE_MTP sub-branch routing to llm_build_gemma4_mtp (new file src/models/gemma4-assistant.cpp); the base gemma4 iSWA path is unchanged.

Everything else is additive (the LLM_GRAPH_TYPE_MTP enum, llm_graph_input_mtp class + set_input/can_reuse, build_attn_mtp, the mtp_slot_info/init_mtp KV machinery, the C API llama_decode_mtp, and the --spec-type draft-assistant arg). Default-inert: the MTP path is reached only with the assistant loaded + --spec-type draft-assistant; plain decode and native draft-mtp are unchanged.

Touches zero .cu/.cuh — the assistant rides the existing fattn/turbo/DCA CUDA kernels, so landing it on the live tree is a cosmocc host rebuild only (no DSO rebuild/restamp). Capture: snapshot-diff from the spike 0078-tip base (23 files, 1 new src/models/gemma4-assistant.cpp, 0 index lines); reverse + forward byte-identical in the spike; spike host build GREEN (compiles + links; strings shows draft-mtp and draft-assistant coexisting). Live-binary composition gates are PENDING (#461): assistant-MTP × {PolyKV / POSITION_WINDOW / turbo / DCA} logit-equivalence (real_frac=0, the locked "non-MTP works in MTP" gate) runs on the live CUDA binary, gated on a host rebuild + GPU availability. Applies after 0078.

Upstream-alignment overlay (#476 — adopt upstream #23398/#24282 DATA-plane, keep our control-plane)

Upstream llama.cpp merged Gemma-4 MTP (PR #23398 = release b9549, + E-series #24282). 0079 now carries upstream's data-plane identity so a future llamafile bump onto a base containing #23398 reconciles the gemma assistant-MTP as a clean delete of the identity layer, keeping only our control-plane delta. The rename folded into 0079 (S1; re-runs of the same hunks, no new files):

surface	was (ours)	now (upstream-aligned)
arch string	gemma4_assistant	gemma4-assistant (hyphen)
proj tensors	mtp.pre_projection / mtp.post_projection	nextn.pre_projection / nextn.post_projection (LLM_TENSOR_NEXTN_PROJ_{PRE,POST})
E-series tensors	mtp.centroids / mtp.token_ordering.weight	masked_embd_centroids / masked_embd_ordering (bare, NO .weight) (LLM_TENSOR_MASKED_EMBD_*)
backbone-width KV	gemma4_assistant.n_embd_backbone	%s.embedding_length_out (LLM_KV_EMBEDDING_LENGTH_OUT); hparams n_embd_out_impl
layers KV	(derived)	adds %s.nextn_predict_layers (= block_count)

What we deliberately do NOT adopt — ctx_other. Upstream's assistant mechanism is a second llama_context that aliases the target KV by struct-copying its layer record + mirroring cell metadata each step (llama-kv-cache.cpp apply_ubatch: if (other) { v_cells = other->v_cells; return; }; seq_*/update all if (other) return;). It assumes the target KV is vanilla: single-owner, contiguous, unquantized — the opposite of opencoti's KV moat, and it is independently broken under quantized KV (public PR: 0% acceptance with -ctk q8_0 -ctv q8_0). It would conflict with all five gate-required features (PolyKV per-stream layers, rolling-KV POSITION_WINDOW, M2/M3 head-split, TurboQuant KV, DCA). So we keep the opencoti control-plane: single-context build_attn_mtp read-only cross-read (ctx_dft == nullptr, load INTO target via llama_model_load_mtp_from_file); ordered/centroid masked-embedder head; tok_embd→OUTPUT buft; fused N-step; --spec-type draft-assistant (CLI string, not GGUF identity — preserves the 2-engine dispatch vs qwen draft-mtp). Bespoke keys with no upstream equivalent (n_centroids, centroid_top_k, use_ordered_embeddings, attention.k_eq_v, requires_target_arch) remain opencoti-private under the gemma4-assistant.* prefix — upstream loaders ignore unknown keys.

The rename is semantically INERT — proven at three levels: (1) the re-formatted A4B drafter GGUF (.opencoti/gguf-upstream-rename.py, Q8_0 bytes copied verbatim) has whole-file tensor-data sha256 == source; (2) live boot-smoke on the renamed CUDA host binary loads gemma4-assistant, routes draft-assistant, drafts (44% accept), decodes coherently; (3) real_frac = 0 logit-equivalence on the renamed engine (.opencoti/s4-p3-logit-equiv.sh, A4B v6-coder Q4_K_M). The 0079 patch is surgically confined: 16 of its 23 file-stanzas are byte-identical to the pre-rename 0079; only the 7-file data-plane surface (llama-arch.{cpp,h}, llama-context.cpp, llama-hparams.h, llama.cpp, common/speculative.cpp, src/models/gemma4-assistant.cpp) changed — the entire control-plane (kv-cache, graph, model registration, server branch, base gemma4) is untouched. Chain proof (G33/G35): 0079 regenerated as diff(clean-0078-replay, live) where the replay is a fresh git archive HEAD + llamafile apply-patches.sh overlay + strict git apply of 0001..0078; the FULL chain 0001..0079 then strict-applies from pristine (0 rejects), final tree == live byte-identical across 2941 source files, reverse round-trips. A4B beachhead only — re-formatting the other 4 drafters (E2B/E4B/12B/31B) + the full composition matrix on the renamed engine are FOLLOW-ON (#461/#476). Touches zero .cu/.cuh (host rebuild only).

#408 OUTPUT-buft placement re-claimed on 0.10.3 (#484). The tok_embd→OUTPUT-buft win listed in the kept control-plane is not free on the bumped base: 0.10.3's ml.create_tensor selects the device buft by the tensor's llm_tensor_info layer, and TOKEN_EMBD classifies LLM_TENSOR_LAYER_INPUT → CPU buft, so the per-model loader fragments the assistant's per-step full-vocab matmul into a CPU island (MTP-on slower than off at short ctx; bug-540). Re-claimed with an additive llama_model_base::create_tensor_output_head(...) (+ml-less overload, src/llama-model.{cpp,h}) that passes dev_output.buft_list in the INPUT slot to force GPU OUTPUT placement, with normal load accounting (NOT TENSOR_DUPLICATED — tok_embd is the sole-use tied output head). Gated: real_frac = 0 (placement is byte-equivalent) + 1.57× at RULER-32k (off 56.6 → on 88.8 tok/s, 87% accept) — exceeds the historical 1.46×. Folded into 0079 (still 23 files; the 3 stanzas llama-model.{cpp,h} + gemma4-assistant.cpp grew, full 0001..0079 chain re-proven from pristine).

PolyKV multi-slot support folded into 0079 (#485). Two surgical hooks in tools/server/server-context.cpp (both // opencoti F5 M6-S4 mtp (#485)) let the Gemma assistant drafter compose with the SharedKVPool multi-tenant path: (1) at the spec-fit pre-measure (:809), spec_assistant_fit skips the standalone draft-llama_context memory probe for the assistant engine (gemma4-assistant legitimately refuses to be a primary -m model, so the probe would log a benign failed to measure draft model memory warning — the source of the earlier bug-550 misdiagnosis); (2) at the assistant-load branch (:940), a boot guard requires --kv-unified when --parallel >1, because the single-context cross-read (build_attn_mtp → build_attn_mha) infers the stream count from the target K's 4th axis (n_stream = k->ne[3], src/llama-graph.cpp:2115) and the per-stream-split KV layout (0031) hands back a multi-stream tensor the single-token draft query cannot be partitioned against (ggml.c:3754 reshape assert). PolyKV forces --kv-unified, so it composes (real_frac=0 + ~93% draft acceptance @ --parallel 2 --kv-unified); the guard turns the unsupported non-unified multi-slot config into a clean boot error instead of a mid-request SIGABRT. Within the existing 0079 23-file surface (the two hooks live in the already-touched server stanza); full 0001..0079 chain re-proven byte-identical from pristine. Non-unified multi-slot (an n_seq-aware assistant draft) is tracked under #469.

0080-mtp-d512-fa — D512/gqa≤2 flash-attn for the Gemma-4 assistant-MTP drafter (#491+#493, bug-567)

The Gemma-4 E4B assistant-MTP drafter's single GLOBAL layer is head_count 4 / head_count_kv 2 → gqa_ratio 2 at head_dim 512, a kernel-coverage hole on the 0.10.3 base: neither the MMA (switch_ncols2, fattn.cu) nor the TILE (switch_ncols2, fattn-tile.cuh) D512 path instantiates ncols2 ∈ {1,2} (gated DV≤256 for binary size), so a D512/gqa≤2 FLASH_ATTN_EXT op aborts at runtime (bug-567). Every other Gemma-4 config is gqa≥4 (A4B/12B/31B + E4B base; A4B globals are head16/kv2 → gqa 8) and unaffected.

#491 was a host stop-gap: a build_attn_mha guard routed the D512/gqa≤2 layer to the existing non-FA explicit-softmax branch (correct, but "fa off" on the hot path). #493 is the real fix — a genuine D512 f16 flash-attn via the VEC kernel — and retires the #491 non-FA route for the f16 case.

Vendored-source markers // opencoti #491 / // opencoti #493 (5 files, all nested; not an opencoti-hook: — additive vendored-source edits, registered here):

• ggml/src/ggml-cuda/template-instances/fattn-vec-instance-f16-f16.cu + …/generate_cu_files.py — DECL_FATTN_VEC_CASE(512, GGML_TYPE_F16, GGML_TYPE_F16) instance + the autogen-emitter line.
• ggml/src/ggml-cuda/fattn-vec.cuh — extern DECL_FATTN_VEC_CASE(512, F16, F16).
• ggml/src/ggml-cuda/fattn.cu — two head-size switches both get the D512 f16 case: the selector (ggml_cuda_get_best_fattn_kernel: Q->ne[0]==512 && gqa_ratio≤2 && gqa_opt_applies && K/V f16 → BEST_FATTN_KERNEL_VEC) and the VEC dispatch (ggml_cuda_flash_attn_ext_vec, FATTN_VEC_CASE(512, F16, F16) before the GGML_ABORT). Patching only the selector makes it pick VEC while the dispatch falls through to the abort → a decode-time fattn.cu fatal (boot is clean; it crashes on the first global-layer decode). Both are required.
• src/llama-graph.cpp — narrow the #491 guard: fa_no_gpu_kernel now requires K/V not f16/f32 (fa_d512_f16_vec), so f16 D512/gqa≤2 flash-attns on-GPU and only quantized/bf16 D512/gqa≤2 (which the assistant drafter never produces) keeps the soft_max fallback.

The VEC kernel tiles Q->ne[1] (cols_per_block 1 decode / 2 prefill+verify), so the single f16 D512 instance is a real flash-attn at any n_q (decode + prefill + the spec-verify batch). logit_softcap is forced off for D∉{128,256} in the VEC kernel — safe because Gemma-4 attn_soft_cap is false.

Gate (.opencoti/m493-fa-d512-gate.sh): E4B -fa on assistant-MTP decodes with no abort (147.86 tok/s); the verified-greedy output is byte-identical to the #491 soft_max path and draft acceptance is unchanged (0.67910 both) — assistant-MTP is exact spec-decode, so a wrong FA cannot corrupt the verified output but would collapse acceptance; equal acceptance + byte-equal prefix ⇒ the VEC FA is numerically correct. A4B (gqa 8, untouched MMA) regresses nowhere. Both DSO paths rebuilt byte-identical (sha256 a8ddcd82). Chain proof: snapshot-diff capture of 5 files (0 new) from the post-0079 base (vendor backup 20260618-054922, a real #491=0/#493=0 post-0079 tree); git apply --check clean + forward-reproduces live byte-for-byte. Touches .cu/.cuh ⇒ CUDA DSO rebuild + dual restamp required to land live. Applies after 0079. Provenance: opencoti-original.

0081-tcq-q6-asym-kv — TCQ KV tiers + q6_0 scalar KV + asymmetric TCQ FA-VEC (#447/#511/#512/#513/#515)

Three additive KV-quant features landed as one snapshot-diff patch (they share the FA-VEC instance surface and a single host enum bump, so they cannot be split without an intermediate GGML_TYPE_COUNT skew):

1. TCQ tiers — GGML_TYPE_TURBO3_TCQ = 48 (3.25 bpv; 512-state Viterbi, k=3/L=9) and GGML_TYPE_TURBO2_TCQ = 49 (2.25 bpv; 256-state, k=2/L=8). Trellis-Coded Quantization: a GPU FWHT-whitened Viterbi-trellis encode (turbo-tcq-cuda.cuh k_set_rows_turbo{3,2}_tcq, dispatched from set-rows.cu)
   • an O(1) in-register codebook decode fused into FA-VEC (fattn-tcq.cuh readers, smem codebook headers tcq-decode-cb.cuh). Codebooks are real-27B-trained (rebuild-free reloadable). 27B KLD 0.0366 at 3.25 bpv beats plain turbo3_0 0.0470 at iso-bits — TCQ earns its keep on the head_dim-128 target.
2. q6_0 — GGML_TYPE_Q6_0 = 50 (6.5 bpv), a scalar KV type ported from Anbeeld/beellama.cpp (⊃ /shared/dev/ik_llama.cpp, block_q6_0 {ggml_half d; uint8_t qh[8]; uint8_t qs[16];} QK6_0=32). 27B KLD 0.01973 == q5_0 — it sits on the K-precision-dominant frontier (asymmetric strong-K combos like q8_0/q6_0 beat symmetric at equal size). GGML_TYPE_COUNT bumps 48 → 51.
3. Asymmetric TCQ — the mixed turbo3_tcq↔turbo2_tcq FA-VEC instances (both directions), strong-K/cheap-V.

9 keeper FA-VEC instances (the Anbeeld set, not the full cross-product, to bound compile blowup): q6_0 ×5 (q6_0/q6_0, q8_0/q6_0, q6_0/q5_0, q6_0/f16, f16/q6_0) + TCQ ×4 (turbo3_tcq/turbo2_tcq symmetric + both asymmetric). All read through GPU FA-VEC — GGML_CUDA_FA_ALL_QUANTS is the shipped default (#531, build-pipeline.ts --fa-all-quants), and for the non-all-quants build the 9 instances are force-compiled by llamafile/build-functions.sh collect_gpu_sources blocks 3b/3c/3d (the #468/bug-339 dlopen-symbol guard: without them the DSO lacks ggml_cuda_flash_attn_ext_vec_case<D,q6_0|tcq,*> and dlopen fails the instant such KV reaches flash-attention; each is [ -f ]-guarded → no-op on a vanilla tree). A TURBO_TCQ_DUMP_ACTS=<path> env hook (#515) dumps post-FWHT whitened activations for rebuild-free codebook retraining (zero overhead unset).

The earlier turbo3_tcq_w 1024-state free-widen tier (#514) was measured inert and removed (#527) — it is not in this patch (no _w enum/instances/kernels).

Vendored-source markers // opencoti #447 / #511 / #512 / #513 / #515 (38 files, all under vendors/sources/llamafile/ — not opencoti-hook: markers; these are additive vendored-source edits, registered here, and do not change the git grep "opencoti-hook:" packages/ docs/ count):

• 9 NEW ggml/src/ggml-cuda/template-instances/fattn-vec-instance-{q6_0-q6_0,q8_0-q6_0,q6_0-q5_0,q6_0-f16, f16-q6_0,turbo3_tcq-turbo3_tcq,turbo2_tcq-turbo2_tcq,turbo3_tcq-turbo2_tcq,turbo2_tcq-turbo3_tcq}.cu.
• 3 NEW ggml/src/ggml-cuda/{fattn-tcq.cuh, tcq-decode-cb.cuh, turbo-tcq-cuda.cuh}.
• Host enum/traits/quants: ggml/include/ggml.h, ggml/src/{ggml.c, ggml-common.h, ggml-quants.c, ggml-quants.h, ggml-turbo-quant.c}, ggml/src/ggml-cpu/{ggml-cpu.c, quants.h}, common/arg.cpp (kv_cache_types += {Q6_0, TURBO3_TCQ, TURBO2_TCQ}), tools/perplexity/main.cpp.
• CUDA glue: ggml/src/ggml-cuda/{common.cuh, convert.cu, cpy.cu, cpy-utils.cuh, dequantize.cuh, fattn.cu (FATTN_VEC_CASES + supports_op), fattn-common.cuh, fattn-vec.cuh, ggml-cuda.cu (SET_ROWS supports_op), set-rows.cu}, ggml/src/iqk/{iqk_common_extra.h, iqk_ggml_type_ext.h}.
• Graph/DCA lift: src/{llama-graph.cpp, dca.cpp} (scalar q6_0 + asym compose with DCA via the #444 f16-dequant-on-lift; TCQ aborts under DCA, rc=134, pending #505).
• Build glue: BUILD.mk, outer llamafile/build-functions.sh.

Correctness via KLD / logit-equiv / RULER niah — never greedy needle (bug-263/270). RULER niah 8 tiers 100/100 @ 4k–512k (#529 §5b). PPL/KLD valid only on Qwen3.6-27B-Omnimerge (head_dim 128); A4B raw-PPL is garbage (bug-631). TCQ pays an in-register-Viterbi decode tax (~15% slower tg on weight-bound 27B, ~45% on small A4B). Chain proof: snapshot-diff capture (38 files, 9 NEW) from the post-0080 BASE (vendor backup 20260622-111006-pre-tcq-q6-asym-capture); applying 0081 to a clean apply(0001..0080) tree byte-identically reproduces GOAL (full-tree diff -rq empty, nested + outer). Touches .cu/.cuh ⇒ CUDA DSO rebuild + dual restamp required to land live. Applies after 0080. Provenance: opencoti-original (q6_0 ported from Anbeeld/beellama.cpp ⊃ ik_llama.cpp).

0089/0093–0095 — bug-858 MTP parity ship + bug-2108 chain repair (2026-07-05, #608/#614)

bug-2108 chain repair (renumbering). The committed chain no longer strict-applied from clean 0.10.3: 0089-rolling-kv-compute-reserve (and later patches) were snapshot-captured from a tree carrying uncaptured work (sparse-attn vslash #551/#557–568 incl. the sparse_attn_block_size KV-ctor plumbing, #591/#592 decode-overhead work, bug-1840 fixes), so their context lines baked in content no patch introduced. Repair: 0089-sparse-attn-vslash-foundation (NEW — diff(chain-0088 baseline, vendor-backup 20260701-123044), proven to be exactly old-0089's capture base); old 0089→0090, 0090→0091; old 0091-rolling-kv-diag-visibility retired (its llama-kv-cache.cpp hunks folded into 0093). Full-chain proof: clean git archive 0.10.3 + Mozilla llama.cpp.patches overlay + strict git apply of all 41 patches ⇒ byte-identical to the live vendored tree (excl. tools/ui/dist fetch-time assets + submodules). Lesson (bug-250 class): run the chain proof at EVERY capture, not once per era.

Markers registered by this ship:

• // opencoti-hook: sparse-attn-vslash (common.h + graph/KV plumbing) — vertical-slash sparse attention foundation; patch 0089.
• bug-858 dual-ctx comment blocks (// opencoti bug-858 dual-context MTP …) across speculative.cpp, llama-context.*, llama-kv-cache*, llama-model.cpp, models/gemma4-assistant.cpp, server-context.cpp — the ctx_other/mem_other+share drafter-shares-target-KV port; patch 0093. Control-plane stays opencoti (--spec-type draft-assistant --mtp-head + OPENCOTI_MTP_DUAL_CTX=1); data plane aligned with upstream #23398/#24282.
• // opencoti-hook: mtp-verify-mmvf (bug-2103) (ggml-cuda/mmvf.cu) — TinyBLAS-gated ne11≤8 mmvf cap for the spec-verify GEMV cliff; patch 0093.
• // opencoti-hook: tinyblas-small-gemm (bug-2104) (llamafile/tinyblas.cu) — second small-GEMM config when the 128×64 config yields <24 blocks; patch 0094. NOTE: outer-llamafile file (not llama.cpp) — survives llama.cpp bumps, conflicts only on llamafile-proper bumps.
• // opencoti-hook: ckpt-buf-noinit (#614/bug-2107) (common.h, server-task.h, server-context.cpp) — common_state_buf + checkpoint buffer recycle pool; rides with checkpoint_min_step 256→8192 (bug-2105, plain default alignment, no marker — registered via this row); patch 0095. Future-bump gotcha (bug-2106): the struct default is header-inlined into EVERY constructing TU (incl. o/llamafile/*.o) — after editing it, purge all constructing objects (both arches, CCACHE_DISABLE=1) and gate on the boot line min spacing = 8192, never on "build succeeded".

0096 — single-context Gemma MTP facade REMOVED (#611, 2026-07-05)

END of the bug-858 port. The dual-context assistant (patch 0093) is now the ONLY --spec-type draft-assistant execution path — the OPENCOTI_MTP_DUAL_CTX env gate and the whole single-context in-target engine are deleted (~1600 lines: decode_mtp* facade/sync/fused + async worker, process_ubatch_mtp, sched_mtp, llama_decode_mtp C API, iswa init_mtp, build_attn_mtp, llm_build_gemma4_mtp, common_speculative_impl_draft_assistant).

Registry deltas: the opencoti F5 M6-S4 mtp markers covering those functions are gone with them; the surviving assistant surface is the loader (llama_model_load_mtp_from_file / mtp_assistant), the dual-ctx graph in models/gemma4-assistant.cpp, and the 0093 dual-ctx comment blocks. The NextN fused-draft markers (fused-nextn-mtp, mtp-verify-mmvf) are untouched. A future llamafile bump reconciles the assistant almost entirely against upstream #23398/#24282 (data plane AND execution shape now upstream-aligned; remaining opencoti-specific: control-plane flags + single-binary loading).

Gate: A4B n2 305.4 tps / 0.849 accept == 0093 reference (boot line dual-context draft created without any env); q35 NextN n3 248.8 / 0.662. bug-2110 (apparent accept 0.635 at --parallel 2 --kv-unified) was RESOLVED same-day as a tps-harness artifact (degenerate-filler prompt): on the real 9-prompt mtp-bench the dual-ctx assistant holds accept at par2 (ours 0.785/0.783 par1/par2; upstream 0.793/0.792) — PolyKV × assistant-MTP composes cleanly on the dual-ctx path.

2026-07-05 — 0097 native-ELF CMake lane (#619, secondary artifact of #613)

0097-native-elf-cmake.patch makes the vendored tree buildable as a standard Linux ELF llama-server (plain cmake + g++/nvcc, rpath libggml-cuda.so, clean gdb/nsys) alongside the untouched cosmocc Makefile lane.

Registry additions (all markers opencoti-hook: native-elf-cmake (#619)):

• llama.cpp/ggml/src/CMakeLists.txt — add ggml-turbo-quant.c, ggml-neo-pipeline.cpp, ggml-iqk-flash-attn.cpp to ggml-base sources.
• llama.cpp/src/CMakeLists.txt — add dca.cpp (matches neither the llama-*.cpp list nor the models/*.cpp glob).
• llama.cpp/common/CMakeLists.txt — add pcie-profile.cpp.
• llama.cpp/ggml/src/ggml-cpu/CMakeLists.txt — add ggml-cpu/opencoti-elf-stubs.cpp (CMake-lane-only).
• llama.cpp/tools/server/CMakeLists.txt — set_source_files_properties(server.cpp … main=llama_server): the Mozilla overlay's server.cpp defines int main where upstream defines llama_server; the cosmo lane renames at compile time, CMake must too.
• NEW FILE llama.cpp/ggml/src/ggml-cpu/opencoti-elf-stubs.cpp — return-false stubs for the llamafile/sgemm.cpp dispatchers (llamafile_mixmul, _mixmul_needs, _mixmul_iqk, llamafile_fa_*); every call site treats false as "use the upstream generic path" (same contract as Mozilla's fa_helpers_unsupported.cpp), so the ELF build loses only CPU fastpaths. sgemm.cpp itself includes <cosmo.h> and stays cosmo-lane-only.

Bump guidance: on a future llamafile bump, re-anchor the five one-line list insertions (trivial context) and keep the stub file as-is unless sgemm.cpp's dispatcher signatures change. The CMake lane REQUIRES -DGGML_CUDA_FA_ALL_QUANTS=ON — upstream's CMake only globs fattn-vec*.cu (our turbo/TCQ instances + codebook setters) under that flag; the cosmo build always sets it. CMake lane uses cuBLAS (not TinyBLAS) — per-toolchain perf gating (bug-2103/2104 class).

Chain state after 0097: 43 patches, strict-apply + byte-identical replay proven from clean 0.10.3 archive + Mozilla overlay on 2026-07-05.

2026-07-05 — 0098 rolling-KV lse-decode (bug-1843, #623)

0098-rolling-kv-lse-decode.patch extends the Stage-3a LSE channel (opencoti_fattn_dst_lse / opencoti_fattn_dst_lse_written, both defined in fattn-common.cuh and drained by launch_fattn) to head_dim<=256 n_q==1 decode at the three streaming-FA call sites in llama.cpp/ggml/src/ggml-cuda/fattn.cu (markers: opencoti-hook: f5-rolling-kv bug-1843 at the S3b overlap site; short bug-1843: notes at the S3a single-stream and S2 resident-view sites). The per-tile streaming_lse_kernel recompute now fires only when opencoti_fattn_dst_lse_written reports that no finalize (stream-k fixup / parallel-blocks combine) ran. No header change; no new symbols; DSO-only rebuild. Chain state after 0098: 44 patches, strict-apply + byte-identical replay proven from clean 0.10.3 + Mozilla overlay (2026-07-05).

2026-07-06 — 0099-0102 DCA-Gemma wiring + attn-rot inherit + streaming scratch ring (bug-2118/2119/2120/2121)

Four fixes captured after the Gemma×DCA contamination discovery (bug-2118) and the two criticals it surfaced:

• 0099-dca-gemma4-wiring.patch — surgical hooks in llama.cpp/src/models/gemma4.cpp
  • gemma4-assistant.cpp: build_attn_inp_dca + per-layer dca_l dispatch to build_attn_dca (GLOBAL layers; assistant Q-only). Marker: opencoti F5 dca.
• 0100-dca-single-chunk-defer.patch — llama.cpp/src/dca.cpp (opencoti-owned file): single-chunk regime defers quant lift like analytical-bands.
• 0101-kv-attn-rot-inherit.patch — llama.cpp/src/llama-kv-cache.cpp ctor share branch inherits attn_rot_k/v + Hadamard tables from the source cache. Marker comment: opencoti bug-2120.
• 0102-fattn-bulk-h2d-scratch-ring.patch — llama.cpp/ggml/src/ggml-cuda/fattn.cu: bug-2116 bulk-H2D quant tail lift + bug-2121 persistent per-slot staging scratch ring (markers: opencoti-hook: f5-rolling-kv bug-2116 / bug-2121). RULE banked in cerebrum: never share ggml_cuda_pool scratch across CUDA streams.

Chain state after 0102: 48 patches; full chain strict-applies from clean 0.10.3 pin (7fca8b2 + nested dbe9c0c8 + Mozilla overlay) and reproduces the live tree byte-identically on all touched files (verified 2026-07-06, capture-b21xx).

2026-07-07 — 0103-0104 rolling-KV CPU-FA probe guard + q6_0/q4_0 FA-VEC cell (bug-2125 / #627)

Two fixes for the DCA + custom-KV boot SIGSEGV / abort found on bs2 at 256k/512k:

• 0103-kv-cpu-fa-probe-guard.patch — llama.cpp/src/llama-kv-cache.cpp: gates the Stage-3d-S2 tactic selector's synthetic CPU flash-attn companion probe (opencoti_fa_compute_probe_ms with on_cpu=true) behind a new host predicate opencoti_cpu_fa_capable_ktype. That probe modeled an unused CPU_FA_TAIL cost; for CUDA-FA-VEC-only KV types (q6_0, turbo{2,3,4,8}_0, turbo{2,3}_tcq) the CPU vec_dot trait is NULL, so building/running the host flash-attn NULL-derefed → SIGSEGV at context init (before any request). The predicate returns 0.0f for custom-K tiers, leaving CPU_FA_TAIL unmodeled — the selector keeps its pure-GPU POSITION_WINDOW default (no CPU fallback: the rolling window stays fully on GPU for every KV tier). Marker comment: opencoti bug-2125. Host-only.
• 0104-fattn-q6q4-vec-instance.patch — llama.cpp/ggml/src/ggml-cuda/fattn.cu
  • new template-instances/fattn-vec-instance-q6_0-q4_0.cu + llamafile/build-functions.sh: adds the q6_0-K / q4_0-V FA-VEC cell (strong-K, cheap-V — the K precision ≥ V invariant) missing from the dispatch matrix, which crashed the D≤256 path (Gemma-4 local/SWA) into GGML_ABORT. Marker: opencoti-hook: q6_0 KV (#627/bug-2125). The reverse q4_0/q6_0 direction is intentionally NOT added (K must stay higher-bit). CUDA DSO change. Mirrors the q6_0-q5_0 pattern (#512).

Chain state after 0104: 50 patches; full chain 0001..0104 strict-applies from clean 0.10.3 pin and reproduces the live tree byte-identically on all four touched files (verified 2026-07-07, capture-b2125). Gates (bs2 256k DCA-on): q6_0 / turbo3_tcq / turbo2_tcq symmetric + q6_0-K/q4_0-V all boot HEALTHY + niah_single_1 = 100.0 (every one previously SIGSEGV/abort at boot).

2026-07-09 — 0105-0107 WS1 asym scalar-K/turbo-V ship (bug-2126/2127/2130, #628/#630)

No new opencode source markers. WorkStream-1 (asymmetric strong scalar-K + cheap turbo/TCQ-V decode) lands entirely in the vendored llama.cpp tree as three additive patches; the packages/ Registry above is unchanged. Split: 0105 DCA per-tensor defer-decouple (bug-2127, dca.cpp, host-only, the ~2× decode win); 0106 scalar-K/turbo-V boot-correctness guards (bug-2126, llama-kv-cache.cpp, host-only, two null-fn-ptr SIGSEGV fixes in the rolling-KV boot probe); 0107 turbo-MMA verify kernels (D128/D256/D512, OPT-IN default-OFF)

• the 16 fused-VEC scalar-K/turbo-V decode instances + fattn.cu selector + llama-graph.cpp fuse path + bug-2130 need_f16 (fattn-mma-f16.cuh) + build-functions.sh default-build inclusion. The dropped WS1 option-2 boot "fuse-autoselect" scaffolding (opencoti_turbo_fuse_autoselect, g_opencoti_asym_fuse_enable[2][2], probe_nq/materialize probe params) is not in any patch — it was stripped at ship (inert, default-fuse-equivalent). The WS1 // opencoti-hook: WS1 … markers live in vendored source only (not counted by the packages/ conformance grep). Per-file registry: the vendors/patches/llamafile/README.md 0105/0106/0107 rows.

turbo-MMA VERDICT (#630, 3-model crossover A4B/omni27/qw35a3): earns no default-on anywhere → shipped OPT-IN default-OFF (Gemma A4B/D256 wins ≤128k, net-loses @256k + un-gatable windowed; Qwen/D128 bit-exact but tps-flat, FFN-bound). Chain state after 0107: 53 patches; full chain 0001..0107 strict-applies from clean 0.10.3 pin and reproduces the intended FINAL tree byte-identically (6 modified + 19 new TUs match; llama-kv-cache.h returns to baseline; zero scaffolding symbols) — verified 2026-07-09, capture-ws1-ship.

2026-07-10 — 0108-0109 WS2 turbo-V in-register DCA-decode ship (bug-2134/2140, #629)

No new opencode source markers. WorkStream-2 (read a rotated turboN V-cache IN-REGISTER inside the multichunk DCA fused flash-attn kernel, killing the per-forward whole-cache cpy_turboN_f16 cast) lands entirely in the vendored tree as two additive patches; the packages/ Registry above is unchanged. Split:

• 0108-ws2-dca-fused-turbo-v-inregister.patch (bug-2134) — llama.cpp/src/dca.cpp (the WS2 in-register gate, now DEFAULT-ON; WS2_DCA_TURBO_V=0 is the f16-lift escape; n_q≤WS2_NQ_MAX(16) gate keeps large-n_q PREFILL on the byte-identical f16-lift path) + ggml/src/ggml-cuda/fattn-mma-f16.cuh (D512+D256 turbo/tcq dca_fused instance macros + the TCQ to_fp16_nc assert exemption for the in-register-V case) + ggml/src/ggml-cuda/fattn.cu (D256 case 256 turbo2_0/turbo3_0/turbo2_tcq/turbo3_tcq V-type dispatch) + template-instances/generate_cu_files.py + the new turbo/turbo-D256/turbo2tcq-D256/ turbo3tcq-D256 dca_fused TU shards. Markers: opencoti-hook: WS2 turbo-V DCA (#629) (host + kernel). bug-455 (dca_fused forces Q_in_reg=false) inherited by the turbo D256 path. CUDA DSO + host change.
• 0109-cuda-dso-2gb-fatbin-last.patch (bug-2140) — llamafile/build-functions.sh
  • llamafile/cuda.sh: two build hooks. cuda-fatbin-last (PRIMARY) — the ~1.9GB .nv_fatbin (6 SM arches × ~260 cubins) sat physically between host .text and its .bss/.got, so every host device-stub PC32/PLT/TLSGD reloc spanned >2^31 → link truncation once the D256 tcq TUs pushed the image past 2GB. Root fix = force .nv_fatbin to the image END via an INSERT AFTER .bss orphan-section linker fragment (pure -Wl, no code-model change; fixes ALL reloc classes; the only fatbin ref left is a 64-bit data pointer). cuda-mcmodel-large (OPT-IN, default none) — OPENCOTI_MCMODEL=large|medium|none forces the large code model on host x86 (compile via nvcc --compiler-options, link via -Xcompiler) + -Xlinker=--no-relax for the prebuilt-CRT/libcudart_static GOTPCRELX; superseded by fatbin-last, kept as a documented escape. Device SASS untouched → zero kernel-perf cost (perf-regression A/B confirms parity). Markers: opencoti-hook: cuda-fatbin-last + opencoti-hook: cuda-mcmodel-large. Build-flag/ linker change only. Per-file registry: the vendors/patches/llamafile/README.md 0108/0109 rows.

WS2 VERDICT (#629, 4-arch × 4-tier matrix @256k DCA-on): default-ON everywhere — in-register turbo-V is a validated decode WIN of +68-95% (D512 A4B/31B, D256 omni27/qw35a3 × turbo2_0/turbo3_0/turbo2_tcq/turbo3_tcq), lossless by RULER niah (100) + teacher-forced logit-equiv (real_frac 0 / ffa 1.0), incl. A4B MTP-on n_q>1 verify (real_frac 0, mean_tv 2e-05). Opposite of the WS1 turbo-MMA null: WS2 removes an O(n_kv) whole-cache cast that grows with ctx, so it wins on the serving decode path. Chain state after 0109: 55 patches; full chain 0001..0109 strict-applies from clean 0.10.3 pin and reproduces the live WS2 tree byte-identically — verified 2026-07-10, capture-ws2-ship.

2026-07-10 — 0110 WS2 asym-band NaN guard (bug-2141, #629/#626)

No new opencode source markers. The WS2 (0108) in-register turbo-V dca_fused decode reads V rotated and de-rotates the graph OUTPUT via wht_o; the normalized O = VKQ / rowsum is produced in the shared stream-k combine. Under DCA-on decode the fused kernel walks INTER→SUCC→INTRA phases under one continuous online-softmax and load-balances via the stock kbc stream-k decomposition, so the final divide-by-rowsum lands in the shared stream-k fixup for seam blocks (and the process_tile complete-tile write-back otherwise). A band-boundary phase with an all-(−inf) mask gives a ZERO exp-sum (rowsum==0) → 0/0=NaN, which wht_o's 128-pt butterfly smears across all 512 de-rotated dims → NaN logits → empty generation. Surfaces ONLY on asymmetric turbo_tcq K≠V (the hi-K/cheap-V serving pattern): t3tcqK/t2tcqV + t2tcqK/t3tcqV pre-fix niah 20 / 4-of-5 EMPTY @256k/8-chunk (already 100 @512k/16-chunk) — a composition no symmetric control exercises. Discriminator WS2_DCA_TURBO_V=0 (V→f16-lift, same turbo3 K-lift) recovers to niah 100 / 0-empty, isolating the fault to the in-register-V + wht_o composition (NOT the K-lift/codebook — all per-tensor type selection verified correct).

• 0110-ws2-asym-band-nan-guard.patch (bug-2141) — guards the zero exp-sum divide at the three FA output-normalization sites feeding wht_o: ggml/src/ggml-cuda/fattn-mma-f16.cuh (flash_attn_ext_f16_process_tile complete-tile write-back) + ggml/src/ggml-cuda/fattn-common.cuh (flash_attn_stream_k_fixup_uniform + flash_attn_stream_k_fixup_general stream-k seam combines). Emits the identity 0 for a fully-masked row; the rowsum!=0 branch runs the EXACT original division (not a reciprocal-multiply) → bit-identical for all symmetric turbo/TCQ + scalar tiers and all non-DCA callers of these shared kernels; the WS2 decode win is untouched (no-op on the hot path). Markers: opencoti-hook: WS2 asym-band NaN guard (bug-2141) (3 sites, kernel-only). CUDA DSO change; no host binary change. Per-file registry: the vendors/patches/llamafile/README.md 0110 row.

RE-GATE VERDICT (#632, fixed DSO f1c554d8): full asym turbo_tcq matrix RECOVERED — t3tcqK/t2tcqV AND t2tcqK/t3tcqV at 256k AND 512k all niah 100.0 / 0-empty (pre-fix t3tcqK/t2tcqV @256k was 20 / 4-of-5 empty). Symmetric controls (turbo2_tcq, turbo3_tcq @256k) hold 100 / 0-empty. WS2 in-register decode win preserved through the guard (A4B turbo2_0 @256k warm ~91.2 tps, uncontended). Off-path bit-identicality confirmed by the symmetric/scalar controls staying 100 plus the guard's rowsum!=0 branch running the identical division. Chain state after 0110: 56 patches; full chain 0001..0110 strict-applies from clean 0.10.3 pin and reproduces the live tree byte-identically — verified 2026-07-10 (fattn-mma-f16.cuh 58cdae34, fattn-common.cuh f7af3a8d).

2026-07-10 — 0111-0113 rolling-KV spill made graceful (bug-2144/2145/2146, #635/636/637/631)

Three host-only additive patches that make rolling-KV overflow (POSITION_WINDOW KV residency) actually graceful — the "spill fallback" now degrades on the PCIe/bytes axis instead of cliffing. All validated on the same binary (#637, Qwen2.5-14B-1M Q8_0 @128k, bs2 RTX 6000, single-GPU, needle PRESENT every cell); the DSO is unchanged (the bug-2144 kernel tail-read src[7]/[8] already shipped in the C2 DSO with 0108). New markers live in vendored source only; the packages/ Registry is unchanged.

• 0111-dca-streaming-tail-bug2144.patch (bug-2144) — llama.cpp/src/dca.cpp. DCA-on rolling-KV spill decode cliffed (~55×) because build_attn_dca_core used the whole-cache get_k/get_v, which for a spilling POSITION_WINDOW layer return the ggml_concat(window, tail) fallback (sync full-KV round-trip every step), bypassing the async streaming path. Fix gives the fused dca_fused kernel a two-region window+tail streaming path (window on GPU, pinned-host tail on src[7]/[8], op_params[5]=n_win), gated dca_window_decode (FA ∧ small-n_q ∧ !single_chunk ∧ !get_is_fragmented() ∧ POSITION_WINDOW active ∧ window active). DEFAULT-ON (escape WS2_DCA_STREAM=0); safe because the master flag only arms the path and the predicate enforces every precondition structurally. Markers: opencoti-hook: DCA streaming-tail (bug-2144-C / Stage C) (2 sites). Host-only.
• 0112-rolling-kv-phasegate-prefill-bug2145.patch (bug-2145) — llama.cpp/src/llama-graph.cpp. The non-DCA streaming-window path has no batched large-n_q kernel → PREFILL ran decode-style over the window = flat ~325 tok/s cliff. Fix gates the window fast-path to decode/small-n_q (WS2_NQ_MAX d16) and routes large-n_q prefill to the amortized whole-cache concat + batched build_attn_mha (path B), at both dispatch sites. Markers: opencoti-hook: rolling-KV phase-gate prefill (bug-2145) (3 sites). Host-only.
• 0113-rolling-kv-reserve-fixpoint-bug2146.patch (bug-2146) — llama.cpp/src/llama-context.cpp. The bug-1342 two-pass reserve measured the compute buffer against a degenerate wc=256 window; with the 0111 streaming decode path that forces a maximal tail whose scratch inflated the measurement (~4088 vs real ~926 MiB → 3.16 GiB over-reserve, a fitting KV pushed to a 3168 MiB tail). Fix iterates the reserve to a fixpoint (cap 3 iters) — each re-size grows the window/shrinks the tail/shrinks the buffer, converging to full residency when the KV fits; non-streaming converges in one step byte-identical to prior. No new marker (the fixpoint loop is self-documented bug-2146). Host-only.

VALIDATED (#637): DCA-on streaming spill tracks-and-beats the DCA-off rolling-KV reference at every matched tail — plateau ~37 tps to a ~4.9 GiB tail, common knee ~5.6 GiB, graceful rolloff, NO cliff; ~17× recovery at a 6.7 GiB tail (0.68→11.69). Reserve fixpoint recovers full residency (37.10→38.40 @vt=41888) and improves the spill case (+62% @vt=35400). Default-on re-gate (no env) reproduces the numbers. Chain state after 0113: 59 patches; full chain 0001..0113 strict-applies from clean 0.10.3 pin and reproduces the live tree byte-identically — verified 2026-07-10 (dca.cpp efa73317, llama-graph.cpp 874fa901, llama-context.cpp e40aa9e2). Per-file registry: the vendors/patches/llamafile/README.md 0111/0112/0113 rows.

2026-07-10 — 0114 hybrid rolling-KV residency + 0111 ggml host-op amendment (bug-2142, #633)

• 0114-hybrid-rolling-kv-residency-bug2142.patch (bug-2142) — llama.cpp/src/llama-memory-hybrid.{h,cpp}, llama-memory-hybrid-iswa.{h,cpp}, llama-model.cpp. The rolling-KV residency knobs (--vram-target, --kv-residency-mode, the bug-1342 compute-reserve, the window-measure-pass) reached the iSWA and plain-dense memory paths (llama-model.cpp create_memory) but were silently dropped for hybrid models: llama_memory_hybrid and llama_memory_hybrid_iswa hardcoded 0/0.0f/false when building their internal attention llama_kv_cache, so a hybrid model (qw35a3 = Qwen3.6-35B-A3B DeltaNet+attn NextN MoE) on a too-small card ignored --vram-target and OOMed. Fix threads the five residency params through both hybrid wrapper ctors to the attention cache, mirroring the iSWA/plain forwarding; the new ctor params default to the prior off-state so any non-forwarding caller and every resident config stays byte-identical. Markers: opencoti-hook: rolling-KV residency for hybrid (bug-2142) (2 create_memory call sites in llama-model.cpp, one per hybrid variant). Host-only — no DSO change. VALIDATED (#633, bs2 GPU0): fixed --vram-target 32000 @ -c 262144 on qw35a3 → host tail = 4705 MiB (knob honored); pre-fix (C2ship) same cmd → FULLY RESIDENT … no host tail (dropped, GPU overshoots the 32000 target to 34907 MiB). Fully-resident decode unchanged (182.5 t/s); spill decode 77.8 t/s (bandwidth-bound, as expected).

• 0111 amendment (bug-2144 completeness). The originally-shipped 0111-dca-streaming-tail-bug2144.patch captured only the dca.cpp call-site (kp_tail/vp_tail/dca_n_win) but NOT the matching host-side op-node construction that call needs to compile — the k_tail/v_tail/n_win parameters on ggml_flash_attn_ext_dca_fused and their src[7]/src[8]/op_params[5] wiring in ggml.h/ggml.c. The shipped C2 binary built from a working tree that had those edits, so a strict re-apply of the chain would REVERT them and fail to compile; the gap was masked because the chain proof did not build. 0111 is regenerated to include the ggml.h/ggml.c hunks (recovered byte-identically from the pre-ship vendor backup). Lesson banked: the chain-apply proof MUST include a build step so a non-building chain can't hide an uncaptured edit. See buglog bug-2142/bug-2147.

Chain state after 0114: 60 patches. Full-chain 0001..0114 strict-apply + build proof is the gating step for this section (in progress at commit time).

2026-07-11 — 0115 refuse context-shift on a spilled rolling-KV window (bug-2148, #638)

• 0115-rolling-kv-shift-guard-bug2148.patch (bug-2148) — llama.cpp/src/llama-kv-cache.cpp, one hunk in llama_kv_cache::get_can_shift(). The F5 M7 Stage-3c position window splits the cache across a GPU-resident window [0,wc) and a CPU-resident tail [wc,kv_size) when --vram-target forces overflow. Context-shift (server seq_rm+seq_add→pos_add) and self-extend (seq_div→pos_div) re-rope the cache IN PLACE via the llm_graph_input_k_shift / build_rope_shift graph, which views only k_per_stream — sized to the window's wc cells, NOT get_size() (llama-kv-cache.cpp ~4720-4744). So the moment a shift fires with a spilled window it over-reads the window tensor and hits ggml.c GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)); and even were the view to fit, the host tail (written already-roped, never transformed again — no CPU-side rope path) would be left stale-roped. Fix: get_can_shift() returns false when any layer has an active spilled window (window_cells > 0 && a populated k_cpu_per_stream); the server reads this at init (common_init_from_params → "KV cache shifting is not supported for this context, disabling KV cache shifting"; also server-context.cpp:1093) and cleanly disables ctx_shift + cache-reuse, so a spilled slot is bounded at n_ctx instead of crashing. Fully-resident windows (wc==kv_size, no tail, window_cells sentinel 0) and every non-window cache find no spilled layer → BYTE-IDENTICAL and still context-shift normally; iSWA (get_can_shift ANDs kv_base+kv_swa) and hybrid (get_can_shift delegates to mem_attn) wrappers propagate the leaf guard. The "no-degradation" alternative (a host-side rope-by-delta pass over the CPU tail on every shift) is DEFERRED — out of scope for the multi-session prefix-shared serving target, which does not rely on infinite-context shift over a spilled tail. Marker: opencoti-hook: rolling-kv-shift-guard (1 site). Host-only — no DSO change. VALIDATED (#638, bs2 GPU0, Qwen2.5-14B-1M-Q8_0, -c 12288 --context-shift --keep 256): windowON (--vram-target 18000, 6912 resident / 5376 host tail) → shift disabled at init, request bounded (predicted_n=3840, 8448+3840=12288), coherent, NO assert; controlOFF (--vram-target 80000, fully resident) → shift STILL fires (n_discard=6015), coherent to 1219 — guard does not over-fire. Prior binary asserted at ggml.c:1840 on the windowON shift. Byte-identical git apply of 0115 == the live edit.

Chain state after 0115: 61 patches.

2026-07-11 — 0116 auto KV-tier selection at boot (HAL #582 P1, #621)

• 0116-kv-auto-tier-hal582-p1.patch (#621) — the HAL #582 P1 policy: on a dense full-attention model, auto-pick the LEAST-compressing scalar KV type pair that keeps the whole cache resident, rather than making the operator hand-tune -ctk/-ctv. New static helper opencoti_auto_select_kv_tier in llama-kv-cache.cpp, called in the llama_kv_cache ctor BEFORE any residency/allocation math consumes type_k/type_v. It reuses the EXACT budget arithmetic of opencoti_compute_resident_window_cells (free − already-resident − compute-reserve, capped by --vram-target) and the same per-layer per-cell byte sum, then walks the ladder {f16, q8_0/q8_0, q8_0/q4_0, q5_1/q4_0} (HAL §2 hi-K/cheap-V recipe; q5_1/q4_0 last-resort floor) and selects the first (least- compressing) rung whose whole-cache cost fits; floor if none fit (rolling-KV then spills the excess). Announced at WARN with a set -ctk/-ctv to override hint (mirrors the sparse-V auto-policy #567). Strictly opt-in (OPENCOTI_KV_AUTO_TIER=1) and a no-op — byte- identical — when the env is unset (every existing caller), there is no GPU offload, the user set -ctk/-ctv off the f16 default (override respected), the model is iSWA (swa_type != NONE, Gemma stays f16/resident per HAL §3c), or f16 already fits.
• Marker: opencoti-hook: kv-auto-tier (1 site — the ctor helper + call in llama-kv-cache.cpp). Provenance: opencoti-original.
• Host adapter opt-in: kvAutoTier config → OPENCOTI_KV_AUTO_TIER env in buildServerArgs (packages/opencoti-llamafile). Additive, no surgical hook.
• Host-only — no DSO change (make-only build, ~11 s ccache-warm). VALIDATED (#621, bs2 GPU0, Qwen2.5-14B-1M-Q8_0, -c 32768, RTX 6000): ladder walks correctly as --vram-target tightens — 20000 MiB → q8_0/q8_0 (KV 3264 MiB fits budget 3328), 19400 → q8_0/q4_0 (2496 fits 2728), 18800 → q5_1/q4_0 (2016 fits 2128); every rung FULLY RESIDENT … no host tail, passkey needle clean. Config-equivalence: auto q8_0/q8_0 output byte-identical to explicit -ctk q8_0 -ctv q8_0, which emitted zero auto-tier WARN (opt-in path confirmed off). Byte-identical git apply of 0116 == the live edit.

Chain state after 0116: 62 patches.

2026-07-11 — 0117 re-rope a spilled rolling-KV window on context-shift (#639, bug-2148 reopened)

• 0117-rolling-kv-shift-rerope-639.patch (#639) — supersedes 0115. 0115 stopped the spilled-window shift from crashing by REFUSING it (get_can_shift() returned false when a layer had an active spilled window), which cleanly but silently disabled ctx_shift AND KV prompt-cache-reuse for any multi-session slot whose KV spilled — the exact serving-path capability this fork exists to preserve. 0117 does the deferred hard part instead: build_graph_shift re-ropes BOTH regions of a spilled window with the same build_rope_shift op — the WINDOW on its GPU tensor k_per_stream[s] (a view bounded to wc cells, so it never over-reads get_size(); shift slice [base, base+wc)) and the TAIL on its host tensor k_cpu_per_stream[s] (a view of the kv_size-wc tail cells; shift slice [base+wc, base+kv_size)). The tail tensor lives in host memory, so ggml's scheduler dispatches its rope op to the CPU backend automatically — identical rope formula, one-shot on shift events only, never per decode token — while attention still runs entirely on GPU over the already-roped window+tail (no CPU-attention / CPU_FA_TAIL divergence class). get_can_shift() returns true again (keeping the STEP35 and n_pos_per_embd()>1 guards); the 0115 refusal loop is removed.
• Regression shield: fully-resident windows (window_cells sentinel 0) and every non-window cache take the windowed==false path; for n_stream==1 the window view is sized get_size() and passes k_shift_src directly — BYTE-IDENTICAL to the pre-0117 op (the n_stream>1 non-windowed view is unchanged too). Host-only, no CUDA DSO rebuild (the tail rope is an existing ggml op dispatched to CPU).
• Markers: opencoti-hook: rolling-kv-shift — 2 sites (get_can_shift, build_graph_shift), RENAMED from 0115's single-site rolling-kv-shift-guard. The 0115 patch file and its historic marker text remain (0117 supersedes its behavior additively; 0115 stays in the chain).
• VALIDATED (#639, bs2 GPU0, clean/no-contention, Qwen2.5-14B-1M-Q8_0, -c 12288 --context-shift --keep 256): spilled (--vram-target 18000, 6912 resident / 5376 host tail = 1008 MiB) → shift ENABLED, coherent through the shift (ZULU-9999 present, monotonic run 1219, output byte-identical to the fully-resident control), NO assert. Decode-tps == pre-#639 baseline within noise (74.88→43.67 over ~2800 tok vs baseline 74.47→40.25); the re-rope is shift-event only, no per-token cost. The 26.9-tps "regression" seen in the first gate was GPU/host contention (bug-2149), not this patch. Byte-identical git apply of 0117 == the live edit.

Chain state after 0117: 63 patches.

2026-07-11 — 0118 T*-aware auto rolling-KV spill (HAL #582 P1+, #621 follow-on)

• 0118-kv-auto-tier-tstar-spill.patch (#621 follow-on) — extends the 0116 auto-tier helper opencoti_auto_select_kv_tier so it can KEEP a higher-quality tier (up to f16) with a small host-tail spill when that spill is provably cheap, instead of always quantizing to the first fully-resident rung. The ladder fit-condition changes from "first tier fully resident" to "first tier whose SPILL ≤ T*_bytes", T*_bytes = (target_drop·t_resident − c_engage)·bw_eff (target_drop default 0.20, env OPENCOTI_KV_TSTAR_DROP; clamp ≥0, ≤ cliff env OPENCOTI_KV_TSTAR_MAX_SPILL_MIB default 800). Boot-measured terms: T1 explicit-axis handling; T2 bw_eff = min(pcie, host_ram) with a NEW host 1-thread memcpy probe (OPENCOTI_KV_HOST_RAM_GBPS); T3 fa_ms_cell+c_engage from the reused FA compute microbench (#351, OPENCOTI_KV_CENGAGE_MS); T4 the drop-model. Fast-path guard keeps the "f16 fits" boot byte-identical; probe-unavailable or T*≤0 degrades exactly to 0116's fully-resident gate; the ctx-value gate (req #2) falls out of the formula (short ctx ⇒ tiny t_resident ⇒ c_engage dominates ⇒ quantize).
• Strictly opt-in (OPENCOTI_KV_AUTO_TIER=1, the same 0116 switch) and a byte-identical no-op when unset. HOST-ONLY: the FA microbench dispatches to the existing CUDA DSO — no ggml-cuda.so rebuild.
• Marker: opencoti-hook: kv-auto-tier — EXTENDS the existing 0116 site (the ctor helper + its one call), no new hook site. The 0116 marker text covers it.
• VALIDATED both hosts (T2-T4 terms fire; solidPC PCIe-bound bw_eff 6.7, bs2 host-RAM-bound 9.3-10.3; drop-model monotone in target_drop). T5 RULER-niah quality/tps gate PASS (bs2 GPU0, Qwen2.5-14B-1M-Q8_0, -c 65536, --vram-target 27000 → ~2 GB f16 spill): auto → q8_0/q8_0 (T* only ~14 MiB), f16-window-spill decode-tps drops 78.5% (≫ 20% target), q8 niah == f16 niah == 100 (lossless), q8 tps 35.3 = 3.3× f16-spill 10.8 — quantize verdict correct, fallback dominates. Byte-identical git apply of the full chain 0001..0118 == the live edit (cmp OK).

Chain state after 0118: 64 patches.

0119-p0-dcaon-scalar-inregister (#582-P0 / #643 + #644)

Scalar q4_0/q4_1/q5_0/q5_1/q6_0/q8_0 K/V read IN-REGISTER inside the multichunk flash_attn_ext_f16_dca_fused MMA kernel, killing the whole-cache to_fp16_nc dequant-lift that dominated quant-KV DCA-on decode (HAL §3b profile ~29.5 s/decode). Scalar generalization of #629's turbo-V in-register read; reuses the SAME load_tile_dequant / get_dequantize_V path (no new kernel math).

• Marker: opencoti-hook: P0 DCA-on scalar in-register — 3 sites: fattn-common.cuh (the 4 new ne==8 V-reader branches, q4_1/q5_0/q5_1/q6_0 — the only correctness surface, bug-2152; the MMA loader stays fixed at ne_per_chunk=8, SHARED with the proven turbo path, NOT changed); fattn-mma-f16.cuh (skip in-launch K materialize for a scalar type_K, mirroring the existing type_V==F16 guard; + D128 scalar externs); fattn.cu (decode-gated dispatch, Q->ne[1] ≤ WS2_NQ_MAX=16, env WS2_DCA_SCALAR_KV default on — matched pair dispatches its typed instance, any other combo falls through to the shipped F16-materialize default).
• 40 NEW template-instance TUs (fattn-mma-f16-dca-fused-scalar-<tag>-d128-{0..3}.cu; 4 ncols2 shards × 10 pairs) — named fattn-mma-*.cu so collect_gpu_sources globs them. D128 only (Qwen full-attention); D256/D512 scalar deferred (Gemma KV is turbo, already in-register).
• Correctness: #643 beachhead q8_0-K/q4_0-V @D128 ctx128k real_frac 0.0 + 2.13× decode (14.72→31.33 tps); #644 all 10 pairs @D128 32k multichunk DCA-on real_frac 0.0. Byte-identical git apply of 0119 onto a reconstructed 0118 baseline == the live edit (cmp OK, 43 files). Additive-only, touches only nested llama.cpp files (no outer-file conflicts). DSO 2.5→3.83 GB, absorbed by the fatbin-last fragment (bug-2140); -mcmodel=large stays OFF. Applies after 0118.

Chain state after 0119: 65 patches.

0120-p2-mixed-kv (#622 / #582-P2)

Position-axis mixed KV for rolling-KV spill: the resident window [0,wc) keeps the boot KV type (e.g. q8_0) while the spilled host tail [wc,n_kv) may carry a MORE-compressed type (e.g. q4_0). Recent (attention-dominant) tokens stay high-fidelity on-device; the older tail pays fewer H2D bytes per decode step. Position-axis complement to the head-axis (#620/#643 P0) and boot-tier (#621 P1) residency levers. New CLI -ctkt/-ctvt set the tail K/V type; sentinel GGML_TYPE_COUNT (flag unset) means "tail == window" → every path is byte-identical to the shipped uniform-window behaviour.

• Marker: opencoti-hook: P2 mixed-kv — additive 5-point plumbing carrying the tail type from CLI to the pinned host allocation: common/arg.cpp (--cache-type-k-tail/-v-tail → common_params.cache_type_k_tail/_v_tail, common/common.{h,cpp}) → llama_context_params (include/llama.h) → llama_cparams (src/llama-cparams.h) → kv-cache ctor (src/llama-kv-cache.{cpp,h}, src/llama-kv-cache-iswa.{cpp,h}, src/llama-context.cpp, src/llama-model.cpp), which allocates k_cpu_per_stream/v_cpu_per_stream at the tail type. Decode (the win): the POSITION_WINDOW streaming FA op (ggml/src/ggml-cuda/fattn.cu) already reads window + tail each in-register at its own type (reuses #620/#629 readers) — no whole-cache f16 materialise.
• Two correctness-fallback fixes in src/llama-kv-cache.cpp, both host-only, both byte-identical when tail type == window type:
  • bug-2161 — the get_k/get_v concat-reassembly accessor (prefill / graph-reserve, n_q > WS2_NQ_MAX) feeds ggml_concat, which asserts a->type == b->type. Fixed with a file-local poswin_lift_to_f16 (mirror of dca_lift_to_f16: f16 passthrough; scalar quants via ggml_cast→F32→F16, byte-exact GPU CPY) applied to both region views only when the two types differ.
  • bug-2162 — the Stage 3c-6 state-IO window paths (K/V × write/read) used the window row size for the tail region, so an in-memory context-checkpoint (llama.cpp PR16391, on by default) asserted on the 2nd prompt at a mixed config. Fixed with a per-region ggml_row_size(k_cpu->type / v_cpu->type, n_embd_*_gqa), symmetric across writer and reader; parity assert recomputed from both region sizes.
• Gate (S7, bs2 Qwen2.5-14B-Instruct-1M-Q8_0, -c 24576 --vram-target 18000, logit-equiv vs full-resident gold; NEVER greedy needle): correctness real_frac 0.0 for U8/MIX/U4, advisory mean-TV U8 0.0069 < MIX 0.0110 < U4 0.0118 (mixed strictly between uniform-q8 and uniform-q4, closer to gold than uniform-q4); throughput MIX 28.1 vs U8 26.4 tps = +6.4% (host tail 11264 cells); regression -ctkt q8_0 -ctvt q8_0 sentinel byte-identical to unset (REG==U8). The gate exercises context checkpoints, so it doubles as a checkpoint round-trip test for bug-2162. Byte-identical git apply of 0120 onto a reconstructed 0119 baseline == the live edit (0 files differ) + clean reverse-apply on the live tree. Additive-only, touches only nested llama.cpp files. Applies after 0119.

Chain state after 0120: 66 patches.

0121-p2-auto-tail (#653 / #582-P2)

P2-auto boot policy: when the P1 boot auto-tier (opencoti_auto_select_kv_tier, 0116/0118) is FORCED to spill — the auto-selected KV window still overflows the VRAM budget, so the rolling-KV path spills a tail regardless — auto-pick a MORE-compressed type for the spilled tail [wc,n_kv) than the resident window, reclaiming H2D bytes on the oldest (least-attended) cells for free. Reuses the 0120 tail plumbing (type_k_tail/type_v_tail, GGML_TYPE_COUNT sentinel = "tail == window"); this patch only adds the boot POLICY that fills those sentinels in.

• Opt-in is SEPARATE from P1: OPENCOTI_KV_AUTO_TIER_TAIL=1 (in addition to OPENCOTI_KV_AUTO_TIER=1). With P1 alone the tail stays == window, so a P1-only boot is byte-identical to before this patch — the compressed tail engages only when the operator also asks for it. An explicit -ctkt/-ctvt (tail != COUNT) is honored and never overridden, mirroring how -ctk/-ctv block the window auto-tier.
• Policy — "q4_0 floor when window > q4_0": for each axis independently, if the chosen window type is higher-bit than q4_0 (f32/f16/bf16/q8_0/q6_0/q5_1/q5_0) and that axis's tail is unset, set the tail to q4_0; otherwise leave tail == window (so a window already at/below q4_0 is not "upgraded" to a coarser tail). The policy hook (maybe_auto_tail) fires ONLY on the two spill exits — the "window kept but spills" T* branch (chosen==0) and the "chosen tier still spills" tail — so it never touches the fully-resident fast path (early return).
• Marker: reuses the existing 0116 opencoti-hook: kv-auto-tier site — this edit lives inside opencoti_auto_select_kv_tier and its one call site, exactly like 0118. No new inline hook site; the 0116 marker text covers it.
• Gate (bs2, Qwen2.5-14B-Instruct-1M-Q8_0, -c 24576 --vram-target 18000, --flash-attn on, logit-equiv vs full-resident gold; NEVER greedy needle): P2AUTO (both envs) boot log shows auto KV tail = q4_0/q4_0 over a real ~189 MiB / 3072-cell spilled region (window q5_1/q4_0 at the second-pass binding budget — bug-1342 two-pass compute reserve runs the auto-tier twice, the first looser pass fits, the binding pass spills), real_frac 0.0, advisory mean-TV 0.0087 (not worse than the uniform-tail P1AUTO floor). P1AUTO (OPENCOTI_KV_AUTO_TIER=1 only) shows NO tail line — regression shield: byte-identical to the shipped P1 boot. HOST-ONLY (src/llama-kv-cache.cpp is libllama in the host binary, not the CUDA DSO): make rebuild, DSO ccache-hits.
• Byte-identical git apply of 0121 onto a reconstructed 0120 baseline == the live edit (cmp OK) + clean reverse-apply. Full chain 0001..0121 strict-applies from clean 0.10.3. Additive-only, touches only the nested llama.cpp/src/llama-kv-cache.cpp. Applies after 0120.

Chain state after 0121: 67 patches.

2026-07-13 — 0122 RYS layer duplication --repeat-layers (#656–#668)

0122-rys-layer-duplication (RYS-1..12)

--repeat-layers SPEC — load-time, weight-shared re-run of source transformer layers in place (RYS / dnhkng "passthrough self-merge" — mergekit passthrough + SOLAR depth-upscaling, done at inference time with no new weights). SPEC = ;-separated i,j blocks re-running layers [i,j) a second time right after the original run; the empty plan ⇒ off ⇒ byte-identical to before this patch (eff == src, identity map). All host-side; no CUDA.

• Data plane. common_params.repeat_layers (parsed in common/arg.cpp) → cparams.layer_plan, a std::vector<uint32_t> of length n_layer_eff where layer_plan[eff] = src (the original layer each effective step re-runs). Parsed + validated in src/llama-context.cpp (opencoti_parse_repeat_layers): fail-loud on out-of-range or malformed spec. The forward graph iterates the EFFECTIVE plan and passes eff (the KV slot) to build_attn; weights/hparams are keyed by src = layer_plan[eff] (RYS off ⇒ eff==src ⇒ every existing lookup is identity).
• KV allocation across all cache families. map_layer_ids + slot allocation extended to n_layer_eff in: the unified cache (llama-kv-cache.{cpp,h}), the iSWA dual-cache (llama-kv-cache-iswa.{cpp,h} — base + swa each eff-length, the is_swa filter resolved via an inner eff→src so a duplicated global stays global), the recurrent/hybrid state caches (llama-memory-recurrent.{cpp,h}, llama-memory-hybrid{,-iswa}.{cpp,h}), and the DCA build path (dca.cpp — eff→src for the single per-source is_swa hparam it reads).
• Per-arch forward loops (src/models/*.cpp): Qwen2/2.5/3 dense, Qwen2-MoE + Qwen3-MoE, Qwen3.5-MoE + Qwen3-Next recurrent (gated-delta-net), and Gemma-4 iSWA dense (12B/31B, per-layer √ embed-scale keyed by src) + A4B (gemma4.cpp). Bucket-C recurrent boot crash root-caused + fixed here (auto-FA/GDN device resolvers parsed the EFFECTIVE idx from the op name → dev_layer(eff) out-of-range; eff→src map at 3 sites, bug-2160).
• Boundary-layer advisory (bug-2164): a boot LLAMA_LOG_WARN when the plan duplicates a layer within the first/last max(3, n_layer/8). Duplicating early/late layers commonly yields incoherent output — this is a MODEL property (franken-merge boundary fragility), NOT an engine bug: verified by a middle/boundary × SWA/global 2×2 on native Gemma-4-A4B via the CHAT endpoint (middle SWA L4/L10 + middle GLOBAL coherent; boundary GLOBAL L29 garbage — garbage tracks boundary-ness, independent of SWA/global). Coherent band ≈ L4..L24 (Gemma-4-A4B), only the last layer on Qwen3-8B. The advisory is the entire "fix"; there is no correctness code change — the engine is provably correct on middle layers.
• Markers: opencoti-hook: rys-layer-dup (vendored-source markers only — additive vendored edits registered here; do not change the git grep "opencoti-hook:" packages/ docs/ upstream-opencode inventory). Provenance: opencoti-original (adopts dnhkng RYS-II + mergekit-passthrough conventions).
• Gates — coherence smoke via the CHAT endpoint (RYS intentionally changes the model, so logit-equiv-vs-baseline is the wrong bar; and the target models are thinking-models whose answer lives in reasoning_content — raw greedy /completion is garbage even at baseline). Qwen3-8B dense off/plain/DCA/vram-window/sparse all coherent; Gemma-4 A4B (MoE-iSWA) + 31B (dense-iSWA) middle-layer dup coherent, boundary dup garbage + advisory fires; RYS × rolling-KV residency window × DCA-on composes coherently on iSWA (window engages, no boot crash); recurrent (qwen35moe GDN) boots + self-stacks including full-model 0,40.
• Byte-identical git apply of 0122 onto a reconstructed 0121 baseline == the live edit; full chain 0001..0122 strict-applies from clean 0.10.3. TS adapter: repeatLayers field/flag in packages/opencoti-llamafile/src/{config,launch}.ts. Applies after 0121. See features/rys_layer_duplication.md.

Chain state after 0122: 68 patches.

2026-07-13 — 0123 RYS template probe --rys-probe (#669–#670)

0123-rys-probe (rys-probe v1.1)

--rys-probe — a new host-only PROGRAM MODE (ProgramMode::PROBE) that maps a GGUF for good RYS layer-duplication templates (companion to the 0122 --repeat-layers engine), the cheap alternative to dnhkng RYS-II's days-long beam + XGBoost-surrogate + graded-benchmark search.

• New standalone file llamafile/rys_probe.cpp (driver, lf::rys_probe_main), detected in llamafile/args.cpp (→ ProgramMode::PROBE, enum in args.h), dispatched from llamafile/main.cpp, built via llamafile/BUILD.mk. Four additive common_params fields (rys_probe, rys_probe_widths="auto", rys_probe_band="auto", rys_probe_topk=10) + four --rys-probe* flags in llama.cpp/common/{common.h,arg.cpp}.
• Mechanism: parse args as the PERPLEXITY example, disable -fit (its trial context aborted the flash-attn print), common_init_from_params(model_only=true) — load ONCE. For each contiguous [i,j) in the band [max(3,n/8) .. n-max(3,n/8)) × widths, rebuild ONLY the llama_context (llama_init_from_model, cp.repeat_layers="i,j"; layer_plan has no runtime setter, weights are plan-independent), score ΔPPL (small corpus, compact batched NLL) + a graded 24-item task-probe (greedy + substring early-stop), free ctx.
• Output: ranked table + exactly two templates — MOST EFFICIENT (max ΔPPL/added-layer)
  • MAX GAIN (lowest ppl); honest "same block is both" when no wider block wins. The task score is both a rank signal and a guardrail (drop candidates below base task). The two templates feed perf/llamafile/rys-probe-eval.sh for the real downstream verdict.
• Markers: none needed — this is a NEW standalone mode, not an invisible behavior change to an upstream file; a run without --rys-probe is byte-identical (the model path is untouched). Provenance: opencoti-original (cheap analogue of dnhkng RYS-II + mergekit passthrough conventions). No TS adapter (a diagnostic CLI, not a runtime server flag).
• Gates: host builds clean (incremental; the common_params field-ADD needs a clean rm -rf o rebuild — bug-2160/2166 ABI-skew class — but the later widths default-VALUE change is ABI-safe). Smoke on Qwen3-4B-Q4_K_M (band [10,16) auto, 14 cand): base ppl 30.15 task 0.917 → winner 11,13 (ppl 28.03, task 0.958). Mapped Qwen3.6-35B-A3B-UD-IQ3_XXS (n_layer 40, band [5,35), 109 cand) on the 3090.
• Byte-identical git apply --reverse --check of 0123 against the live tree (hunks match exactly); common.h/arg.cpp baseline reconstructed from the post-0122 pre-rys-probe vendor-backup, the 5 outer llamafile/* files diffed against HEAD (== post-0122; no prior patch touches them). Full-chain 0001..0123 strict-apply from clean 0.10.3 = Phase B (post-map, via build:llamafile:check). Applies after 0122. See features/rys_probe.md.

Chain state after 0123: 69 patches.

2026-07-13 — 0124 RYS×MTP draft-context guard (bug-2171)

0124-rys-mtp-ctx-guard (bug-2171)

RYS --repeat-layers must not propagate into an MTP/NextN draft context. With RYS active and self-speculation on (--spec-type draft-mtp, Qwen NextN or the gemma4 assistant), the draft llama_context inherited the target's params.repeat_layers (it is built from common_context_params_to_llama(params_base)), so its cparams.layer_plan was RYS-duplicated too. But the NextN head sits at the canonical il = n_transformer (n_layer - nextn_predict_layers), which the RYS eff→src map never contains → boot aborts at draft-context init with cpy_k: map_layer_ids MISS il=n_transformer (n_keys=0) → failed to create MTP context. RYS is a target-only inference capability: in self-spec the NextN head consumes the target's already-RYS hidden state through shared memory, so the drafter must run the plain base layer stack.

• Fix (src/llama-context.cpp, in the RYS plan-expansion block of the ctor): gate the spec on context type — const char * rys_spec = (cparams.ctx_type == LLAMA_CONTEXT_TYPE_MTP) ? nullptr : params.repeat_layers; then parse that (nullptr ⇒ empty plan ⇒ eff==src identity for the draft). The target context still receives params.repeat_layers unchanged, so RYS on the target is untouched; only the draft context is forced plain. sched_reserve() mirrors the same guard so the reserve graph matches.
• Lossless: greedy self-spec has the target verify every drafted token, so the drafter running the plain stack (vs an RYS-duplicated one) changes only acceptance/tps, never output. Boot now clean: RYS 5,9 + draft-mtp on Qwen3.6-35B-A3B boots with RYS … 44 effective layers and ctx_dft=yes (no MISS).
• Byte-identical when RYS is off (empty repeat_layers ⇒ the guard picks nullptr either way) and when MTP is off (target path unchanged). Marker: reuses the 0122 opencoti-hook: rys-layer-dup site (same feature, no new hook). HOST-ONLY (no CUDA). Provenance: opencoti-original. Applies after 0123. See features/rys_layer_duplication.md.

Chain state after 0124: 70 patches.

2026-07-14 — 0125 hybrid order-agnostic rolling-KV window (bug-2172 / #672)

0125-hybrid-window-orderagnostic (bug-2172 / #672)

Order-agnostic rolling-KV position-window scatter for a HYBRID model's attention sub-cache. On qwen35moe / qwen3next (gated-delta-net recurrent + full-attention + MoE + NextN), the hybrid memory is llama_memory_hybrid wrapping a llama_kv_cache (mem_attn, !is_recurrent) and a llama_memory_recurrent (mem_recr). When rolling-KV arms its position-window on mem_attn under tight VRAM / large ctx, the recurrent batch pipeline (split_seq + [TAG_RECURRENT_ROLLBACK_SPLITS] + find_slot cell-reuse/wrap) hands the attention scatter non-monotone destination cell idxs, which trip the append-only window scatter's monotone GGML_ASSERT in cpy_k (llama-kv-cache.cpp:4083, pre-patch).

• Fix (host-only; single TU src/llama-kv-cache.cpp + a one-line setter in src/llama-kv-cache.h + a one-line call in src/llama-memory-hybrid.cpp): a new llama_kv_cache::append_only flag (default true) is set false only by llama_memory_hybrid on its mem_attn. Every dense/iSWA/unified cache keeps append_only=true → the append-only fast path is byte-identical + unchanged. When false and in window mode:
  1. the resident window (k/v_per_stream) and host tail (k/v_cpu_per_stream) each get +1 "trash" sink cell (wc_alloc / tail_c_alloc); the append-only path never touches it (reads/writes only [0,wc) / [0,tail_c)), so its compute is byte-identical;
  2. build_input_k/v_idxs emit a 2-row [n_tokens,2] routing tensor (row 0 = window-route → dest cell or window-trash; row 1 = tail-route → dest−wc or tail-trash), filled by set_input_k/v_idxs. ne[0] stays n_tokens so every ->ne[0]==n_tokens graph guard holds;
  3. cpy_k / cpy_v scatter the full per-stream K/V into BOTH window and tail via two arbitrary-dest ggml_set_rows (non-sorted dests are legal → the monotone assert is gone), off-target copies landing in the never-read trash cell. Same op-count at reserve and live (exactly 2 set_rows/stream) so the reserve topology matches.
• Read side UNCHANGED: get_k/v_window reads [0,min(n_kv,wc)), get_k/v_tail reads [0,n_kv−wc) — these already cover every active scattered cell, and the streaming-FA KQ_mask -INF-masks the rollback holes by absolute key position. Mask contract verified: GGML_OP_STREAMING_FLASH_ATTN is fully mask-driven, per-cell, absolute-key-position-indexed, softmax-denominator-normalized → zero CUDA change, no DSO rebuild (host make only).
• Scope: iSWA-hybrid is out of scope (no current model is both hybrid AND SWA; the window never arms on swa_type!=NONE). Documented as such.
• VALIDATED (solidPC host make): qwen35moe auto-spill boots with NO assert + correct output; dense (Qwen) + iSWA (Gemma) regression shields byte-identical (append_only path untouched). Snapshot-diff capture (3 files) from a reconstructed post-0124 baseline; byte-identical-verified (git apply onto 0124-state == live, all 3 files sha256-match). Marker: opencoti-hook: hybrid-window-orderagnostic (3 sites: the append_only flag + the cpy_k / cpy_v scatter branches). HOST-ONLY (no CUDA). Provenance: opencoti-original. Applies after 0124. See features/rolling_kv.md.

Chain state after 0125: 71 patches.

0126-p0-dcaoff-scalar-mma-decode (#582-P0 / #620, #644-mirror)

DCA-off scalar-quant MMA decode. Single-token quant-KV decode with DCA OFF was stuck on the FA-VEC reader, which HAL §3b measured as kernel-bound (~2× f16 regardless of bytes moved — the read never became bandwidth-bound). f16 at high-GQA / long-KV already falls to the tensor-core MMA path (the fattn.cu selector); this patch gives a scalar-quant K/V cache the SAME route: it reads the raw quant blocks straight into the MMA smem tile in-register (flash_attn_ext_f16_load_tile_dequant / get_dequantize_V — no whole-KV to_fp16_nc materialise) and runs the tensor-core mul_mat. Quant decode therefore becomes bandwidth-bound and BEATS f16 (fewer KV bytes moved). This is the DCA-off mirror of 0119's DCA-on scalar in-register read; the two share the same in-register reader family (#629/#630).

• --fa-all-quants premise: opencoti's CUDA build passes --fa-all-quants (build-pipeline.ts → -DGGML_CUDA_FA_ALL_QUANTS), so a native FA-VEC instance for every scalar pair already EXISTS. #620 is an MMA fast-path OVER that VEC path, NOT new support. The env OPENCOTI_QUANT_MMA_DECODE (default-on) selects it; env=0 reverts each pair to its native FA-VEC path (graceful fall-through, never a boot refusal). Scalar KV is not FWHT-rotated, so there is no wht_o / de-rotate (unlike turbo #630).
• Matrix (D128 served pairs, mirrors the #644 scalar set): hi-K/cheap-V {q8K/{q4,q5,q6,q8}V, q6K/q4V} + symmetric diagonal {q4_0, q4_1, q5_0, q5_1, q6_0}.
• Mechanism (all additive; nested llama.cpp only): fattn.cu — 10 forward-decls + octi_quant_mma_decode_enabled() (env gate) + the shared predicate octi_scalar_mma_pair_d128(K,V) (the served D128 scalar pair set); an MMA-dispatch block (OCTI_SCALAR_MMA_D128 macro over the 10 pairs) in ggml_cuda_flash_attn_ext_mma_f16; guard admittance + a VEC-selector escape in ggml_cuda_get_best_fattn_kernel, both gated on dst->src[7]==nullptr (DCA off), so an admitted pair falls straight to BEST_FATTN_KERNEL_MMA_F16. 10 new template-instances/fattn-mma-f16-scalar-<K>-<V>-d128.cu typed instances, each with an entry + ncols1/ncols2 gqa dispatch calling ggml_cuda_flash_attn_ext_mma_f16_case<128,128,…,GGML_TYPE_<K>,GGML_TYPE_<V>>. Auto-collected by build-functions.sh's fattn-mma-*.cu glob (#531).
• VALIDATED (bs2 GPU1, Qwen2.5-14B-1M Q8_0, ctx 65536, n_kv≈44k, correctness via logit-equiv real_frac, never greedy needle — bug-263/270): beachhead q8_0-K/q4_0-V 70.45 tps vs f16 60.99 (1.15×) vs native VEC 30.60 (2.30× — the §3b tax fully killed), real_frac 0.0 vs BOTH f16 and native VEC (top-token identical, lossless). Generalization (5-config @ ctx 65536): decode tps rises monotonically as V gets cheaper — f16 62.15 < q8q4 68.47 < q8q8 69.45 < q6q4 70.31 < q4q4 73.97, all boot coherent, real_frac 0.0 vs f16. Snapshot-diff capture (11 files: 1 mod fattn.cu + 10 new TUs) from a reconstructed post-0125 baseline (the pre-#620 fattn.cu, verified 4 hunks all octi_scalar_mma); byte-identical-verified (git apply reverse-check on live PASS + forward base+0126 == live, all 11 files cmp-identical). Marker: opencoti-hook: P0 DCA-off scalar-quant MMA decode (3 sites in fattn.cu: the predicate + MMA-dispatch block; the guard admittance; the selector escape). CUDA-only (host binary unchanged). Provenance: opencoti-original. No TS adapter (kernel-routing env, default-on; no user-facing server flag — same as 0119). See evaluations/hal_kv_compression.md. Applies after 0125.

Chain state after 0126: 72 patches.

0127-p0-dcaoff-scalar-mma-decode-d256-d512 (#582-P0 / #674)

DCA-off scalar-quant MMA decode — D256/D512 generalization. 0126 (#620) gave a scalar-quant K/V cache the tensor-core MMA decode fast-path on D128 (Qwen full-attention). This patch generalizes that same fast-path to Gemma-4's two head-dims: D256 (SWA-local layers) and D512 (global layers — the KV that grows with context, where the HAL §3c/§3d −28…−35 % kernel-bound quant tax bit hardest). A single Gemma boot now routes a scalar-KV decode through the tensor cores on every layer via the per-layer head-dim predicate. Making Gemma scalar decode bandwidth-bound (so it beats f16) is what makes scalar-quant KV the preferred high-VRAM serving path over turbo — turbo is slower and decisively lower quality.

• Mechanism (0126's, widened; all additive, nested llama.cpp only): the shared predicate octi_scalar_mma_pair_d128 → octi_scalar_mma_pair now admits symmetric head-dim K==V ∈ {128,256,512} (was 128-only); the forward-decls become an OCTI_DECL_SCALAR_MMA(kk,vv) macro emitting all three head-dims; the OCTI_SCALAR_MMA_D128 dispatch macro → OCTI_SCALAR_MMA(kk,vv,base) routing on d ∈ {128,256,512} to base##_d{128,256,512}. The two consult-sites (the --fa-all-quants dead-code guard admittance + the selector escape, both dst->src[7]==nullptr) already read only that predicate, so they generalize for free. 20 new template-instances/fattn-mma-f16-scalar-<K>-<V>-d{256,512}.cu (D256/D512 twins of the 0126 D128 TUs; ggml_cuda_flash_attn_ext_mma_f16_case<256,256,…> / <512,512,…>). D256/D512 MMA + in-register scalar V-readers (ne==8) are proven by the pre-existing turbo D256/D512 instances (#630) and f16's switch_ncols2<256,256>/<512,512>. Auto-globbed (#531).
• VALIDATED (bs2 GPU1, Gemma-4 A4B native google 26B-A4B-128e Q4_K_M, DCA-off, correctness via logit-equiv real_frac vs f16 — Gemma A4B PPL/KLD is garbage, so correctness is top-token identity; never greedy needle):
  • ctx 64k (nkv 42.5k): f16 162.13 < q4q4 170.17 < q8q8 173.86 < q8q4 174.94 tps (q8q4 +7.9 % over f16; env=0 VEC fallback 122.17 = the §3c kernel-bound tax → MMA is +43 % over VEC, tax fully erased). real_frac 0.0 for q8q4/q8q8/q4q4 (lossless).
  • ctx 128k (nkv 112k): f16 152.15 < q4q4 152.95 < q8q4 154.46 tps (q8q4 +1.5 %; §3c had q8q4-VEC at −35 % here). real_frac 0.0 for q8q4/q4q4 (lossless at depth). Win narrows deep as MoE-FFN + per-step launch overhead dilute the KV-bandwidth delta, but scalar stays ≥ f16 and top-token-lossless at both depths.
• Snapshot-diff capture (21 files: 1 mod fattn.cu + 20 new TUs) from the 0126-level fattn.cu extracted byte-exact from the 20260714-042251-pre-0126-620-mma-capture vendor-backup tarball (verified 19 D128 refs; diff-vs-live = exactly the #674 generalize); byte-identical-verified (base+0127 == live, all 21 files cmp-identical). Marker: opencoti-hook: #582-P0 DCA-off quant-MMA decode #620 D128 + #674 D256/D512. CUDA-only. Provenance: opencoti-original. No TS adapter (kernel-routing env, default-on; same as 0119/0126). See evaluations/hal_kv_compression.md. Applies after 0126.
• Build note: the 20 new (esp. D512) instances grow ggml-cuda.so past 4 GiB (~4.43 GiB), which tripped a latent build-pipeline bug (bug-2178): sha256OfFile read the whole DSO into one ArrayBuffer and aborted (V8 max typed-array length), skipping the side-load mirror. Fixed separately in packages/opencoti-llamafile/src/build-pipeline.ts (stream the hash — O(1) memory). The DSO link itself holds under the #629 -mcmodel=large mitigations.

Chain state after 0127: 73 patches.

2026-07-14 — 0128 MTP spec-cycle host parity (bug-858 / #675)

• // opencoti-hook: mtp-verify-mmvf (bug-858/#675) (ggml-cuda/mmvf.cu, 2 sites) — TinyBLAS-gated src0_small && ne11 <= 8 lift in the f16 + bf16 ampere_mma branches of ggml_cuda_should_use_mmvf (siblings of 0093's f32 site, bug-2103/#609). Upstream caps these at ne11==1 because its cuBLAS fallback wins past that; TinyBLAS builds have no cuBLAS, so MTP verify batches (ne11 = n_accepted+1 = 2..4) dropped the thin bf16 shared-expert gate GEMVs (ffn_gate_inp_shexp) onto a single-block tinyblasGE cliff. Base decode (ne11==1) and cuBLAS builds unchanged. Patch 0128.
• // opencoti-hook: spec-clone-cheap (bug-858/#675) (common/sampling.cpp) — common_sampler_clone clones with empty cur/cur_p instead of deep-copying the n_vocab-sized (~1.8 MiB) scratch every speculative cycle: under the cosmo dlmalloc runtime that copy is a fresh mmap + fault + munmap = 1.071 ms/cycle (the entire measured spec wall-clock gap vs b9859, whose glibc reuses the block). cur is write-only scratch (set_logits() repopulates before any read); the copied cur_p.data aliased the source sampler's buffer anyway. Output bit-identical on greedy; clone 1.071 → 0.005 ms/cycle. Sync note: on future bumps, re-check that upstream hasn't started reading a clone's cur/cur_p before set_logits() — if it has, restore the copy (and pool it instead). Patch 0128.
• Gate: Qwen3.6-35B NextN n3 282.6 vs b9859 287.4 (−1.7%, was −12%); base parity; accept bit-identical. Full matrix + 9-prompt bench: evaluations/three-way-tps.md "2026-07-14 refresh".

Chain state after 0128: 74 patches.

2026-07-14 — 0129 rys-probe nested capture + 0130 parallel≥2 boot/checkpoint robustness (bug-2182/2183)

• 0129-rys-probe-nested-args — no new markers; completes the 0123/#669 rys-probe capture: the nested common/common.h common_params fields (rys_probe*) and common/arg.cpp --rys-probe* add_opt registrations were live in the tree but missing from the chain (a fresh apply would fail to compile the outer rys_probe.cpp). Chain-integrity only; found by the 0130 per-file replay (baseline ≠ live on common.h). The rys hunks sit inside regions already marked by 0123's rys-probe comments.
• 0130 bug-2182 (src/llama-kv-cache.cpp) — static tie_scalar_f32() helper + all 20 dep-tie accumulator sites in the opencoti cpy_k/cpy_v scatter (M2 per-stream, hybrid order-agnostic acc2, append-only windowed accumulate/emit) now collapse every tie operand to exactly one F32 element (view ONE FULL BLOCK → cast F32 → view 1 element). Previously the operand ne was the SOURCE blck_size (32 for q8_0/turbo) → add([1],[32]) failed ggml_can_repeat at -fit/graph-build under quant KV × parallel≥2 non-unified. All inside opencoti-owned scatter code (no new upstream-facing marker; the enclosing blocks carry the existing per-stream/rolling-KV hooks). f16/f32 KV graphs byte-identical.
• 0130 bug-2183 (common/common.h, common/common.cpp, tools/server/server-context.cpp) — common_prompt_checkpoint::load_tgt/load_dft return bool instead of GGML_ABORTing when llama_state_seq_set_data_ext returns 0 (legitimate under --kv-unified --parallel≥2 with another slot mid-decode); the server task-launch restore site wipes the seq (tgt+dft) and falls back to full re-prefill via the existing do_reset path; the speculative rollback/draft sites keep fail-fast aborts. Comment-tagged opencoti bug-2183 at all sites. Sync note: upstream declares these loaders void — on future bumps re-apply the bool signature + the server fallback branch together (they are one unit; a void loader with the fallback branch dead-codes the recovery, an aborting loader with the branch reintroduces the crash).
• Gate: 3090, exact bs2 crash configs — q8 KV × --parallel 2 non-unified × -fit on boots n_slots=2 + decodes; forced cell-allocation failure under kv-unified parallel-2 + prompt-cache → 0 aborts, FAILED→re-process fallback served all requests.

Chain state after 0130: 76 patches.

2026-07-15 — 0131-0133 introspection + version string + chain repair (c3)

chain repair (bug-2188)

A faithful full replay (clean 0.10.3 archive + Mozilla llama.cpp.patches overlay + strict git apply 0001..0130) broke at 0124: its second hunk duplicate-added the Mozilla CPU auto-FA block already present from the overlay (the original capture used an ad-hoc "reconstructed 0123 baseline" that lacked it). The redundant hunk was dropped from 0124 (functional RYS guard hunk untouched). The same replay also surfaced llamafile/rys_probe.cpp v2 (#673) shipped live but never captured — now 0131. Whole chain 0001..0133 strict-applies from clean 0.10.3 and reproduces the live tree byte-identically (DIFF_LINES=0). Rule (bug-2108/bug-2188): capture baselines come from .opencoti/s5-build-0078-replay.sh, never ad-hoc reconstructions.

0131-rys-probe-task-battery-v2 (#673)

llamafile/rys_probe.cpp — Δtask-ranked balanced battery (see patches README row). opencoti-owned file (introduced by 0123); no new upstream hook.

0132-runtime-introspection (#676/#677)

Markers: // opencoti #676 / // opencoti #677 comments at every touched site.

• llama.cpp/include/llama.h — struct llama_opencoti_kv_info + llama_memory_opencoti_kv_info() (additive API, after llama_memory_can_shift).
• llama.cpp/src/llama-memory.h — default-false virtual opencoti_kv_info() on llama_memory_i.
• llama.cpp/src/llama-kv-cache.{h,cpp} — impl reading live tensor types + window_cells residency.
• llama.cpp/src/llama-kv-cache-iswa.{h,cpp} — forward to non-SWA base, is_iswa=true.
• llama.cpp/src/llama-memory-hybrid.{h,cpp} — forward to mem_attn.
• llama.cpp/src/llama-context.cpp — C wrapper.
• llama.cpp/tools/server/server-context.h — server_context_meta.json_opencoti (sleep-safe /props).
• llama.cpp/tools/server/server-context.cpp — get_opencoti_props(), get_meta() init, /props key, slot::to_json opencoti block, lifetime draft counters (bug-2187).
• llamafile/BUILD.mk — bug-2186: server.cpp.o rule now depends on tools/server/*.h (stale-object ABI-skew guard). Sync note: on a future llamafile bump, re-anchor the llama.h API block and the server-context meta field/aggregate-init position; the virtuals are additive.

0133-opencoti-version-string (#613 / c3)

• llamafile/version.h — OPENCOTI_TAG default + OPENCOTI_VERSION_STRING (bump the default together with vendors/llamafile/OPENCOTI_TAG on every packaged cut).
• llamafile/main.cpp — --version + help banner print llamafile vX.Y.Z (opencoti-X.Y.Z-<tag>); llamafile vX.Y.Z token must stay FIRST (scripts/llamafile-version.sh contract).
• llamafile/BUILD.mk — OPENCOTI_TAG_FLAG injection (make var from build pipeline) + version.h dep on main.o.