patches/0104-fattn-q6q4-vec-instance.patch

opencoti patch 0104 — q6_0-K / q4_0-V FA-VEC instance (bug-2125 / #627)

q6_0-K + q4_0-V (strong-K, cheap-V — the design invariant K precision >= V) is a
first-class asymmetric KV combo. Under GGML_CUDA_FA_ALL_QUANTS the kernel
selector admits K!=V and picks the VEC path, but no (q6_0, q4_0) FA-VEC instance
existed, so the D<=256 dispatch (Gemma-4 local/SWA layers — the global layers
use the DCA fused kernel's in-launch to_fp16_nc) fell through to
GGML_ABORT("fatal error") at ggml_cuda_flash_attn_ext_vec, crashing at boot.

Fix (mirrors the q6_0-q5_0 pattern, #512/#81): add the instance TU
fattn-vec-instance-q6_0-q4_0.cu (head_dim 64/128/256), the
FATTN_VEC_CASES_ALL_D(Q6_0, Q4_0) dispatch case, and the build-functions.sh 3c
default-build inclusion (so the symbol is present without FA_ALL_QUANTS too).
Verified bs2 256k DCA-on: boots HEALTHY (was abort) + niah_single_1 = 100.0.
CUDA DSO change (needs a cuda rebuild). Only the q6_0/q4_0 direction is added —
reverse q4_0/q6_0 is intentionally NOT added (K must stay the higher-bit type).
--- a/llama.cpp/ggml/src/ggml-cuda/fattn.cu
+++ b/llama.cpp/ggml/src/ggml-cuda/fattn.cu
@@ -353,13 +353,17 @@
     // ik_llama.cpp ships for q6_0 KV), registered OUTSIDE the FA_ALL_QUANTS block so they compile in
     // the DEFAULT build — the matching instance .cu files are added unconditionally in
     // collect_gpu_sources (build-functions.sh section 3c), like the turbo K==V instances below.
-    // q6_0-q6_0 is the symmetric (default-reachable) cell; the 4 mixed cells are reachable only under
+    // q6_0-q6_0 is the symmetric (default-reachable) cell; the 5 mixed cells are reachable only under
     // FA_ALL_QUANTS (the K!=V early-out in ggml_cuda_get_best_fattn_kernel), same as q5_0's mixed pairs.
     FATTN_VEC_CASES_ALL_D(GGML_TYPE_Q6_0, GGML_TYPE_Q6_0)
     FATTN_VEC_CASES_ALL_D(GGML_TYPE_Q8_0, GGML_TYPE_Q6_0)
     FATTN_VEC_CASES_ALL_D(GGML_TYPE_Q6_0, GGML_TYPE_Q5_0)
     FATTN_VEC_CASES_ALL_D(GGML_TYPE_Q6_0, GGML_TYPE_F16)
     FATTN_VEC_CASES_ALL_D(GGML_TYPE_F16,  GGML_TYPE_Q6_0)
+    // opencoti-hook: q6_0 KV (#627/bug-2125) — strong-K q6_0 + cheap-V q4_0 asymmetric cell. Under
+    // FA_ALL_QUANTS the selector admits q6_0-K/q4_0-V, so the stock FA-VEC dispatch (local SWA layers)
+    // needs this instance; without it the D<=256 dispatch fell through to the GGML_ABORT below.
+    FATTN_VEC_CASES_ALL_D(GGML_TYPE_Q6_0, GGML_TYPE_Q4_0)
 
     // opencoti-hook: turboquant perf-lift (Level B) — F5 M6-S2 #392. Fused turbo K/V vec path
     // (always registered, independent of GGML_CUDA_FA_ALL_QUANTS). turbo2/3/4 at head_dim ∈
--- /dev/null
+++ b/llama.cpp/ggml/src/ggml-cuda/template-instances/fattn-vec-instance-q6_0-q4_0.cu
@@ -0,0 +1,11 @@
+// opencoti-hook: q6_0 KV (Anbeeld/beellama.cpp port) — strong-K + cheap-V asymmetric cell (#627/bug-2125).
+// q6_0-K / q4_0-V is a first-class KV combo (6-bit K precision, 4-bit V footprint); the local SWA layers
+// read it through the stock FA-VEC path (the global layers use the DCA fused kernel's to_fp16_nc dequant),
+// so this instance must exist or ggml_cuda_flash_attn_ext_vec_case<D,q6_0,q4_0> is missing and the D<=256
+// dispatch aborts at fattn.cu (GGML_ABORT "fatal error"). Mirrors fattn-vec-instance-q6_0-q5_0.cu.
+
+#include "../fattn-vec.cuh"
+
+DECL_FATTN_VEC_CASE( 64, GGML_TYPE_Q6_0, GGML_TYPE_Q4_0);
+DECL_FATTN_VEC_CASE(128, GGML_TYPE_Q6_0, GGML_TYPE_Q4_0);
+DECL_FATTN_VEC_CASE(256, GGML_TYPE_Q6_0, GGML_TYPE_Q4_0);
--- a/llamafile/build-functions.sh
+++ b/llamafile/build-functions.sh
@@ -162,7 +162,8 @@
     fi
 
     # 3c. opencoti-hook: q6_0 KV (Anbeeld/beellama.cpp port). q6_0 is a scalar KV type read by the
-    #     FA-VEC path; its 5 Anbeeld instances must compile in the DEFAULT build (same #468/bug-339
+    #     FA-VEC path; its 6 instances (5 Anbeeld + q6_0/q4_0 strong-K/cheap-V, #627) must compile in
+    #     the DEFAULT build (same #468/bug-339
     #     reasoning as the turbo block above — the default build does NOT glob fattn-vec, so without
     #     this the DSO lacks ggml_cuda_flash_attn_ext_vec_case<D,q6_0,*> and dlopen fails the instant
     #     q6_0 KV reaches flash-attention). No-op under FA_ALL_QUANTS=1 (already globbed) and on a
@@ -171,6 +172,7 @@
         for f in "$ti_dir"/fattn-vec-instance-q6_0-q6_0.cu \
                  "$ti_dir"/fattn-vec-instance-q8_0-q6_0.cu \
                  "$ti_dir"/fattn-vec-instance-q6_0-q5_0.cu \
+                 "$ti_dir"/fattn-vec-instance-q6_0-q4_0.cu \
                  "$ti_dir"/fattn-vec-instance-q6_0-f16.cu \
                  "$ti_dir"/fattn-vec-instance-f16-q6_0.cu; do
             [ -f "$f" ] && CUDA_SOURCES="$CUDA_SOURCES $f"