opencoti patch 0101 — shared draft cache inherits attention-rotation state (bug-2120) llamafile 0.10.3 attention rotation (Hadamard-rotate K/V rows before quantized store when the cache type is quantized && head_dim%64==0; readers rotate Q and un-rotate the output via attn_inp_k_rot/v_rot) is computed per-cache from its OWN type_k/type_v and accumulated head dims. The dual-context assistant draft cache is created with f16 types AND its shared layers skip the head-dim accumulation, so rotation silently disabled: the drafter read the target's ROTATED quant K/V with an unrotated Q -> decorrelated attention, flat finite logits, accept ~0.000 with ANY quant target KV (f16/f16 unaffected; Qwen NextN unaffected — it reads through the target's own cache object). Fix: in the ctor share branch, inherit attn_rot_k/v, n_embd_head_*_all and the Hadamard tables from the source cache, overriding the local computation. The generic iswa build_attn then applies the rotation on the drafter's Q-only path. Gate (bs2, A4B + assistant Q8_0 drafter, 32k, 24k prompt): accept k8_v16 0.000 -> 0.641; k8_v4 (production PolyKV) 0.000 -> 0.729; f16 control 0.677 unchanged. --- a/llama.cpp/src/llama-kv-cache.cpp +++ b/llama.cpp/src/llama-kv-cache.cpp @@ -1106,6 +1106,26 @@ } } + // opencoti bug-2120: a shared (dual-context drafter) cache must READ with the same + // attention-rotation state the SOURCE cache used to WRITE. The local computation above + // sees this cache's own type_k/type_v (f16 for the drafter) and n_embd_head_*_all == 0 + // (shared layers `continue` before accumulating), so it always disables rotation — the + // drafter then reads Hadamard-rotated quant K/V with an unrotated Q and un-unrotated + // output: decorrelated attention, accept ~0 with ANY quant target KV (f16/f16 unaffected + // because no rotation is applied at all). Inherit flags + head-dim extents + Hadamard + // tables from `other` so build_input_k_rot/v_rot emit the matching rotation inputs. + if (share && other) { + attn_rot_k = other->attn_rot_k; + attn_rot_v = other->attn_rot_v; + n_embd_head_k_all = other->n_embd_head_k_all; + n_embd_head_v_all = other->n_embd_head_v_all; + attn_rot_hadamard = other->attn_rot_hadamard; + if (attn_rot_k || attn_rot_v) { + LLAMA_LOG_INFO("%s: shared cache inherits attn_rot_k=%d attn_rot_v=%d from source (bug-2120)\n", + __func__, (int) attn_rot_k, (int) attn_rot_v); + } + } + const char * LLAMA_KV_CACHE_DEBUG = getenv("LLAMA_KV_CACHE_DEBUG"); debug = LLAMA_KV_CACHE_DEBUG ? atoi(LLAMA_KV_CACHE_DEBUG) : 0; }