# Kimi K3 FlashKDA candidate runbook

This is a **candidate-only** prefill experiment.  It is not included in the
baseline serving build and is disabled in the candidate binary unless the
environment switch below is set.

## Build

The candidate source tree has the adapter under
`ggml/src/ggml-cuda/flashkda-k3/`.  Build it with CUDA 13 and the checked-out
FlashKDA source tree:

```bash
cmake -S . -B build-flashkda-candidate \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.0/bin/nvcc \
  -DGGML_CUDA=ON -DGGML_RPC=ON -DGGML_NATIVE=ON \
  -DGGML_CUDA_K3_FLASHKDA=ON \
  -DK3_FLASHKDA_DIR=/home/acer_gn100/flash-kda-source-20260801
cmake --build build-flashkda-candidate -j 12
```

## Guardrails

The bridge is selected only when all of these are true:

- `GGML_CUDA_K3_FLASHKDA=1` is in the **candidate server and every candidate
  RPC worker environment**.  The GDN/KDA CUDA op executes on the worker that
  owns each layer, so setting this only on the head does not select the
  bridge.  The worker's `libggml-cuda.so` must come from the bridge-enabled
  candidate build (verify its marker strings before launch);
- KDA width is 128;
- input tensors have the direct K3 `[D,H,T,N]` contiguous layout;
- the operation has more than one token (prefill only);
- it needs one terminal recurrent state, not GDN history snapshots; and
- it is not using llama.cpp's fused history-cache output path.

Single-token decode remains on llama.cpp's existing fused GDN CUDA kernel.
Any unsupported call falls back to the existing operator.

For a distributed A/B, keep production RPC workers (port 50052) untouched.
Deploy the bridge-enabled `ggml-rpc-server` and CUDA library under a separate
candidate directory on each rank, launch only the candidate workers (port
50053) with `LD_LIBRARY_PATH` pointing there and both
`GGML_CUDA_K3_FLASHKDA=1` and `GGML_CUDA_K3_FLASHKDA_DIAG=1`, and capture each
worker's stderr.  The `K3_FLASHKDA_ACTIVE`/`K3_FLASHKDA_FALLBACK` marker is
printed by the worker, not by the API head.

## Layout contract

FlashKDA declares Q/K/V/G as `[H,T,D]` but supplies stride `(D,D*H,1)`.  Its
raw bytes are therefore identical to GGML's `[D,H,T]`, so those tensors are
not transposed.  Beta is different: GGML stores it as `[1,H,T]` (`h + H*t`),
while FlashKDA uses compact `[H,T]` (`h*T + t`).  The bridge repacks beta.

FlashKDA's output calculation uses BF16 staging.  The adapter then recomputes
the terminal carry from the original FP32 K/V/log-decay/beta tensors, preserving
the GGML FP32 recurrent state for the next prompt chunk.

## Required gates before promotion

1. Start the candidate only during an exclusive maintenance window; a second
   372-GB model instance cannot coexist with the current production instance.
2. Run the 256-token exact-output/state test and retain its output.
3. Compare deterministic candidate vs baseline continuations at temperature 0
   across the held-out short, natural, and long-prefix suites.
4. Benchmark raw uncached 256/512/1024-token prefills plus C1 decode.  Decode
   must remain unchanged; candidate must improve prefill enough to justify its
   BF16 output-path drift.
5. Verify multi-chunk continuation and state reset/reuse behavior before any
   normal API exposure.