patches/0103-kv-cpu-fa-probe-guard.patch

opencoti patch 0103 — rolling-KV CPU-FA cost-probe guard (bug-2125)

The Stage-3d-S2 rolling-KV tactic selector runs a synthetic CPU flash-attn
"companion probe" (opencoti_fa_compute_probe_ms with on_cpu=true) purely to
MODEL a CPU_FA_TAIL cost for its audit log. CPU_FA_TAIL is gated out of
auto-select (#354) — the tail always streams on the GPU (POSITION_WINDOW) — so
the probe's result is never used for the decision.

That probe builds a real ggml_flash_attn_ext over the actual cache K/V types and
runs it on the host CPU backend. For opencoti's CUDA-FA-VEC-only KV types (q6_0,
turbo{2,3,4,8}_0, turbo{2,3}_tcq) the CPU vec_dot trait is NULL, so the CPU
flash-attn dereferences a NULL fn-ptr → SIGSEGV at context-init (before any
client request). Stock types (f16/q8_0/q4_0/q5_0/q5_1/bf16) carry a CPU vec_dot
and survive, which is why only the custom-K tiers crashed under DCA at 256k/512k.

Fix: gate the CPU probe on a host-CPU-FA-capability predicate
(opencoti_cpu_fa_capable_ktype) so it is never built for a K type the CPU
cannot process. Returning 0.0f just leaves CPU_FA_TAIL unmodeled; the selector
keeps its GPU-stream default. The rolling window stays a pure-GPU path for every
KV tier — no silent CPU fallback. Host-only change (no CUDA DSO impact).
--- a/llama.cpp/src/llama-kv-cache.cpp
+++ b/llama.cpp/src/llama-kv-cache.cpp
@@ -2795,6 +2795,28 @@
 // build_attn_mha FA contract (post-permute q[hd,n_q,n_head], k/v[hd,n_kv,n_head_kv],
 // F16 mask). Never aborts boot: every shortfall (no GPU, unsupported op, non-GQA
 // geometry, alloc miss) returns 0.0f and the selector keeps the stream default.
+//
+// opencoti bug-2125 SAFEGUARD: can the HOST CPU flash-attn path
+// (ggml_compute_forward_flash_attn_ext_f16) process a cache of this K type? It reads K via the
+// K-type's CPU vec_dot trait, which is NULL for opencoti's CUDA-FA-VEC-only KV types (q6_0,
+// turbo{2,3,4,8}_0, turbo{2,3}_tcq — GPU in-register readers, no CPU vec_dot / dp kernel). Running
+// CPU flash-attn over such a K deref's a NULL fn-ptr → SIGSEGV. The rolling-KV tail is ALWAYS
+// streamed on the GPU (POSITION_WINDOW; CPU_FA_TAIL is gated out of auto-select, #354), so the ONLY
+// place a host CPU flash-attn is ever built is the Stage-3d-S2 cost-model probe below — and its
+// result is audit-log only. Gate the CPU probe on this predicate so the rolling window stays a
+// pure-GPU path with NO silent CPU fallback for ANY KV tier. Stock ggml types (which DO carry a CPU
+// vec_dot) are the allow-list; every opencoti custom KV type is excluded.
+static bool opencoti_cpu_fa_capable_ktype(enum ggml_type t) {
+    switch (t) {
+        case GGML_TYPE_F32:  case GGML_TYPE_F16:  case GGML_TYPE_BF16:
+        case GGML_TYPE_Q4_0: case GGML_TYPE_Q4_1:
+        case GGML_TYPE_Q5_0: case GGML_TYPE_Q5_1:
+        case GGML_TYPE_Q8_0:
+            return true;  // stock ggml types: have a host CPU vec_dot
+        default:
+            return false; // opencoti custom KV (q6_0 / turbo* / turbo*_tcq): CUDA-only, NULL CPU vec_dot
+    }
+}
 static float opencoti_fa_compute_probe_ms(
         const llama_model & model, const llama_hparams & hparams,
         int32_t il, ggml_type type_k, ggml_type type_v, int64_t bench_cells,
@@ -2823,6 +2845,15 @@
         return 0.0f; // device-type mismatch; selector falls back (stream default / no CPU model)
     }
 
+    // opencoti bug-2125 SAFEGUARD: never build a host CPU flash-attn over a K type the CPU cannot
+    // process (NULL vec_dot → SIGSEGV). This is audit-only (the tail always streams on GPU), so
+    // returning 0.0f just leaves CPU_FA_TAIL unmodeled — the selector keeps the GPU-stream default.
+    // Keeps the rolling window a pure-GPU path for q6_0 / turbo* / turbo*_tcq (and any future
+    // CUDA-only KV tier), with no silent CPU fallback.
+    if (on_cpu && !opencoti_cpu_fa_capable_ktype(type_k)) {
+        return 0.0f;
+    }
+
     const int64_t hd_k      = (int64_t) hparams.n_embd_head_k(il);
     const int64_t hd_v      = (int64_t) hparams.n_embd_head_v(il);
     const int64_t n_head    = (int64_t) hparams.n_head(il);