opencoti patch 0103 — rolling-KV CPU-FA cost-probe guard (bug-2125) The Stage-3d-S2 rolling-KV tactic selector runs a synthetic CPU flash-attn "companion probe" (opencoti_fa_compute_probe_ms with on_cpu=true) purely to MODEL a CPU_FA_TAIL cost for its audit log. CPU_FA_TAIL is gated out of auto-select (#354) — the tail always streams on the GPU (POSITION_WINDOW) — so the probe's result is never used for the decision. That probe builds a real ggml_flash_attn_ext over the actual cache K/V types and runs it on the host CPU backend. For opencoti's CUDA-FA-VEC-only KV types (q6_0, turbo{2,3,4,8}_0, turbo{2,3}_tcq) the CPU vec_dot trait is NULL, so the CPU flash-attn dereferences a NULL fn-ptr → SIGSEGV at context-init (before any client request). Stock types (f16/q8_0/q4_0/q5_0/q5_1/bf16) carry a CPU vec_dot and survive, which is why only the custom-K tiers crashed under DCA at 256k/512k. Fix: gate the CPU probe on a host-CPU-FA-capability predicate (opencoti_cpu_fa_capable_ktype) so it is never built for a K type the CPU cannot process. Returning 0.0f just leaves CPU_FA_TAIL unmodeled; the selector keeps its GPU-stream default. The rolling window stays a pure-GPU path for every KV tier — no silent CPU fallback. Host-only change (no CUDA DSO impact). --- a/llama.cpp/src/llama-kv-cache.cpp +++ b/llama.cpp/src/llama-kv-cache.cpp @@ -2795,6 +2795,28 @@ // build_attn_mha FA contract (post-permute q[hd,n_q,n_head], k/v[hd,n_kv,n_head_kv], // F16 mask). Never aborts boot: every shortfall (no GPU, unsupported op, non-GQA // geometry, alloc miss) returns 0.0f and the selector keeps the stream default. +// +// opencoti bug-2125 SAFEGUARD: can the HOST CPU flash-attn path +// (ggml_compute_forward_flash_attn_ext_f16) process a cache of this K type? It reads K via the +// K-type's CPU vec_dot trait, which is NULL for opencoti's CUDA-FA-VEC-only KV types (q6_0, +// turbo{2,3,4,8}_0, turbo{2,3}_tcq — GPU in-register readers, no CPU vec_dot / dp kernel). Running +// CPU flash-attn over such a K deref's a NULL fn-ptr → SIGSEGV. The rolling-KV tail is ALWAYS +// streamed on the GPU (POSITION_WINDOW; CPU_FA_TAIL is gated out of auto-select, #354), so the ONLY +// place a host CPU flash-attn is ever built is the Stage-3d-S2 cost-model probe below — and its +// result is audit-log only. Gate the CPU probe on this predicate so the rolling window stays a +// pure-GPU path with NO silent CPU fallback for ANY KV tier. Stock ggml types (which DO carry a CPU +// vec_dot) are the allow-list; every opencoti custom KV type is excluded. +static bool opencoti_cpu_fa_capable_ktype(enum ggml_type t) { + switch (t) { + case GGML_TYPE_F32: case GGML_TYPE_F16: case GGML_TYPE_BF16: + case GGML_TYPE_Q4_0: case GGML_TYPE_Q4_1: + case GGML_TYPE_Q5_0: case GGML_TYPE_Q5_1: + case GGML_TYPE_Q8_0: + return true; // stock ggml types: have a host CPU vec_dot + default: + return false; // opencoti custom KV (q6_0 / turbo* / turbo*_tcq): CUDA-only, NULL CPU vec_dot + } +} static float opencoti_fa_compute_probe_ms( const llama_model & model, const llama_hparams & hparams, int32_t il, ggml_type type_k, ggml_type type_v, int64_t bench_cells, @@ -2823,6 +2845,15 @@ return 0.0f; // device-type mismatch; selector falls back (stream default / no CPU model) } + // opencoti bug-2125 SAFEGUARD: never build a host CPU flash-attn over a K type the CPU cannot + // process (NULL vec_dot → SIGSEGV). This is audit-only (the tail always streams on GPU), so + // returning 0.0f just leaves CPU_FA_TAIL unmodeled — the selector keeps the GPU-stream default. + // Keeps the rolling window a pure-GPU path for q6_0 / turbo* / turbo*_tcq (and any future + // CUDA-only KV tier), with no silent CPU fallback. + if (on_cpu && !opencoti_cpu_fa_capable_ktype(type_k)) { + return 0.0f; + } + const int64_t hd_k = (int64_t) hparams.n_embd_head_k(il); const int64_t hd_v = (int64_t) hparams.n_embd_head_v(il); const int64_t n_head = (int64_t) hparams.n_head(il);