docs: model families (read first) + artifact variants §1.1

USAGE.md

@@ -14,6 +14,55 @@ inference server. Deep-dive design docs live in
 
 ---
 
+## Supported / target model families — read this first
+
+opencoti-llamafile loads **any GGUF that upstream llama.cpp 
+loads** — that part is inherited unchanged. But the opencoti feature
+set (KV tiers, rolling-KV, DCA, MTP, sparse attention, RYS) is
+developed, tuned, and correctness-gated on **two model families**, in
+a deliberate primary/secondary split:
+
+### Gemma-4 — PRIMARY target
+
+| Model | Kind | Notes |
+|---|---|---|
+| Gemma-4 26B-A4B-128e ("A4B") | MoE | the flagship serving target; MTP-validated with its native gemma4-assistant drafter |
+| Gemma-4 12B / 31B | dense | full feature validation incl. RYS + DCA |
+| Gemma-4 E2B / E4B | elastic (E-series) | shared-KV elastic layers supported; MTP drafters available |
+
+Gemma-4 is what the engine is *for*: its unusual head dims (256 and
+512), the iSWA sliding/global dual KV cache, and the per-size
+[gemma4-assistant MTP drafters](https://huggingface.co/ManniX-ITA)
+all have dedicated kernels and graph paths here that upstream lacks
+or handles slowly. `--spec-type draft-assistant`, D256/D512 FA-VEC +
+scalar-MMA decode, iSWA-aware rolling-KV/SharedKVPool/DCA wiring are
+all Gemma-4-first features.
+
+### Qwen — SECONDARY target / verification family
+
+| Model | Kind | Notes |
+|---|---|---|
+| Qwen3.5 / Qwen3.6 (e.g. 35B-A3B) | dense / MoE / hybrid (gated-delta-net) | NextN self-spec MTP (`--spec-type draft-mtp`, no external drafter needed) |
+| Qwen2.5-14B-1M | dense, native-1M | the long-context/DCA validation vehicle |
+
+Qwen is the standard-architecture (head_dim 128) counterweight:
+every feature that ships is verified on it too, and it carries one
+feature Gemma doesn't — **NextN self-speculation** (the model's own
+MTP head drafts; fused multi-step, at/above upstream parity).
+
+### Everything else
+
+Other architectures run with upstream behavior and safe fallbacks,
+but opencoti features are **unvalidated** there, and some are
+arch-gated: MTP needs NextN tensors (Qwen-style) or a
+gemma4-assistant drafter; RYS `--repeat-layers` supports the
+qwen2/qwen3(+MoE)/qwen3.5/qwen3next/gemma-4 forward loops; DCA is
+validated on Gemma-4 and Qwen2.5-1M. Quality gates (KLD,
+RULER-niah) were run on the two families above — re-gate before
+trusting aggressive KV tiers on anything else.
+
+---
+
 ## 1. Relationship to upstream llamafile
 
 opencoti-llamafile is **upstream llamafile 0.10.3 plus an additive
@@ -31,9 +80,10 @@ patch series** (`patches/` in the HF repo,
    decode is verified-lossless (the output *is* the target model's).
 3. **Single file, zero dependencies.** The artifact is a Cosmopolitan
    APE: one file runs on Linux/macOS/Windows/BSD, x86_64 and aarch64.
-   The CUDA backend (`ggml-cuda.so`) is embedded and self-extracts to
-   `~/.llamafile/v/<ver>/` on first GPU run. TCQ codebooks and
-   quantization tables are compiled in. No installer, no downloads.
+   In the full x86_64 artifact the CUDA backend (`ggml-cuda.so`) is
+   embedded and self-extracts to `~/.llamafile/v/<ver>/` on first GPU
+   run; the `-win` variant ships without it (see §1.1). TCQ codebooks
+   and quantization tables are compiled in. No installer, no downloads.
 
 What upstream gives you is unchanged: the server API
 (`/completion`, `/v1/chat/completions`, `/props`, `/slots`, …), GGUF
@@ -52,6 +102,41 @@ sh ./opencoti-llamafile-0.10.3-c3-x86_64.llamafile --server --port 8080 \
 > **Note (Linux):** launch via `sh ./file.llamafile` if your kernel
 > lacks binfmt_misc APE registration.
 
+### 1.1 Artifact variants — which file to download
+
+| Artifact | Size | Runs on | GPU story |
+|---|---:|---|---|
+| `…-x86_64.llamafile` | ~4.8 GB | Linux/macOS/BSD, x86_64 + aarch64 | CUDA DSO **embedded** (x86_64-linux, sm_75/80/86/89/90/120f); self-extracts on first `-ngl` run |
+| `…-win-x86_64.llamafile.exe` | ~55 MB | **Windows** + Linux/macOS/BSD, x86_64 + aarch64 | none embedded — CPU works everywhere out of the box; GPU via side-load (below) |
+| `…-aarch64.llamafile` *(planned)* | — | same platforms | CUDA sbsa DSO embedded (sm_110f Jetson Thor, sm_121a DGX Spark GB10) |
+
+The host binary inside every variant is byte-for-byte the same APE
+(same patches, same features); they differ only in the embedded GPU
+backend. The `-win` variant exists because **Windows refuses to run
+executables larger than 4 GB**, so the full artifact — 4.8 GB with
+the CUDA DSO inside — cannot ship for Windows; the bare APE renamed
+to `.exe` (Windows needs the extension) can.
+
+**GPU with the `-win`/bare variant:**
+
+- **Windows:** install the NVIDIA CUDA Toolkit + MSVC and run with
+  `-ngl 99`; llamafile compiles a native `ggml-cuda.dll` on the fly
+  into `%USERPROFILE%\.llamafile\` on first use (upstream llamafile
+  mechanism, unchanged).
+- **Linux x86_64:** download the published standalone DSO
+  (`dso/<ver>-<tag>/ggml-cuda-x86_64.so` in the HF repo — same bytes
+  as the one embedded in the full artifact, sha in
+  `releases/<tag>/SHA256SUMS`) and place it where the engine looks
+  before attempting extraction:
+
+  ```bash
+  mkdir -p ~/.llamafile/v/0.10.3/
+  cp ggml-cuda-x86_64.so ~/.llamafile/v/0.10.3/ggml-cuda.so
+  ```
+
+- **No CUDA available:** nothing to do — it falls back to CPU
+  inference automatically.
+
 ---
 
 ## 2. Feature map — what exists and how it's gated