opencoti F5 M7-A (0034) — pinned host buffer for M2 CPU-half KV
opencoti note (2026-05-31, bug-250): hunk regenerated via snapshot-diff against the post-0033 bootstrap so it strict git-applies — the original removed two trailing-whitespace lines absent from the current tree. Content unchanged (pinned cpu_buft resolution); verified against the live r2a tree.


Switches the M2 head-split CPU-half allocation from
ggml_backend_cpu_buffer_type() (pageable) to ggml_backend_dev_host_buffer_type(dev)
(CUDA pinned host memory) when the layer is offloaded to a CUDA device. Falls
back to pageable CPU buft when no CUDA dev / no dev_host_buffer_type.

Why: pageable host memory caps PCIe transfers at ~50% of theoretical
bandwidth because every DMA has to first pin the page or stage through a
driver-side bounce buffer. Pinned host memory unlocks full PCIe bandwidth
on every M2 CPU→GPU stream-back AND is the foundation Rolling KV (F5 M7)
builds on — without pinned residency, the M7 streaming pipeline can't hit
its compute/copy-overlap budget at PCIe 3.0 x8.

Mechanics:
  - Layer scope (~line 270): resolve cpu_buft alongside the existing
    layer GPU buft. Read dev_host_buffer_type(dev); fall back to plain
    cpu_buffer_type if unsupported.
  - Per-stream scope (~line 391): consume the layer-scope cpu_buft
    instead of calling ggml_backend_cpu_buffer_type() inline.

f16 and CPU-only paths byte-identical to pre-0034 (dev_host_buffer_type
returns null → fallback fires → identical alloc). q8_0 + M2 + CUDA paths
get pinned residency. Foundation for #296 (Rolling KV / M7).

Verification:
  - Cosmocc build clean (~80s incremental).
  - Smoke at 32k + f16 + M2 frac=0.5 + M0/M1/M3 flags engaged on Gemma 4
    A4B Q4_K_M: server boots clean, /v1/completions returns coherent
    tokens (no crash, no DSO mismatch).
  - Does NOT fix bug-226 (256k+q8_0 semantic collapse — confirmed
    deterministic via 1-sample canary 2026-05-29 21:51, scored 0/1 with
    same wrong output 'LJXLD' as the original --no-kv-unified failure;
    pinning is orthogonal to the corruption mechanism).
  - Held for bug-226 root cause before shipping as default — pinning a
    broken stack doesn't fix it but doesn't make it worse either.

Patch order: applies after 0033-cpy-tie-blck-align. Cosmocc-only edit;
CUDA DSO unchanged (host-side buffer-type resolution, no kernel changes).

diff --git a/llama.cpp/src/llama-kv-cache.cpp b/llama.cpp/src/llama-kv-cache.cpp
--- a/llama.cpp/src/llama-kv-cache.cpp
+++ b/llama.cpp/src/llama-kv-cache.cpp
@@ -267,11 +267,20 @@ llama_kv_cache::llama_kv_cache(
 
         ggml_backend_buffer_type_t buft = ggml_backend_cpu_buffer_type();
 
+        // opencoti F5 M7-A (0034) — resolved at layer scope alongside the
+        // per-layer GPU buft so the M2 CPU-half allocation below (line ~390)
+        // can reach pinned host memory when CUDA is offloaded. Falls back to
+        // pageable CPU buft when no CUDA dev / no dev_host_buffer_type. Pinned
+        // host buffers DMA at full PCIe bandwidth; pageable hits ~50%.
+        ggml_backend_buffer_type_t cpu_buft = ggml_backend_cpu_buffer_type();
         if (offload) {
             auto * dev = model.dev_layer(il);
             buft = ggml_backend_dev_buffer_type(dev);
 
             dev_name = ggml_backend_dev_name(dev);
+
+            ggml_backend_buffer_type_t pinned = ggml_backend_dev_host_buffer_type(dev);
+            if (pinned) cpu_buft = pinned;
         }
 
         LLAMA_LOG_DEBUG("%s: layer %3d: dev = %s\n", __func__, il, dev_name);
@@ -374,7 +383,12 @@ llama_kv_cache::llama_kv_cache(
             // when gpu_heads == 0 so headinfer_split_active() can test
             // by vector size.
             if (gpu_heads > 0) {
-                ggml_context * ctx_cpu_s = ctx_for_buft(ggml_backend_cpu_buffer_type(), s);
+                // opencoti F5 M7-A (0034) — was ggml_backend_cpu_buffer_type()
+                // (pageable). Now uses the layer-scope cpu_buft resolved above:
+                // pinned host buffer when CUDA is the layer dev, plain CPU buft
+                // otherwise. Pinned unlocks full PCIe DMA for M2's CPU-half
+                // stream-back AND is the foundation Rolling KV (M7) builds on.
+                ggml_context * ctx_cpu_s = ctx_for_buft(cpu_buft, s);
                 if (!ctx_cpu_s) {
                     throw std::runtime_error("failed to create CPU ggml context for headinfer kv cache");
                 }