# VRAM Calculator — v1 Plan

Pre-flight VRAM predictor + `llama-server` command-builder. Single-GPU and
mixed-GPU. Hero use case: "I want 1M context on a model trained for 262k —
give me the command and tell me it'll still be coherent."

This document is the spine agreed in the /grilling session. The grilling is
closed; everything below is decided.

## Objective

**Maximize quality at a VRAM ceiling.** Quality proxy = highest-bpw quant
that fits. Floor = the 4-bit class (q4_0 / rocmfp4 / nvfp4, ~4.0+ bpw).
Sub-4-bit quants (2-bit/3-bit family) are offered only on explicit user
override and flagged "you asked for it." Backend-agnostic.

## The fit loop (core)

```
target  = user_target_n_ctx
yarn    = auto_configure_yarn(training_ctx, target)   # rope_freq_scale
fit     = max_context(arch, floor_quant, gpus)       # hard VRAM ceiling
result  = min(target, fit)
if fit < target:  present "best you can do = fit, because <reason>"
if extension_ratio crosses ladder: warn (never hard-block)
```

Says "no" cleanly when impossible at floor + hardware, and offers the
floor-quant max_context as the honest fallback.

## YaRN coherence ladder (warnings only, never a hard block)

Monotonic — more extension always means same-or-more concern, never goes quiet:

- **≤ 2× extension** — fine, no warning.
- **> 2× up to 8×** — "expect some quality degradation at the tail."
- **≥ 8× extension** — "likely incoherent at long range; not recommended."

Source: YaRN paper behavior + community experience, not measured. Extension
ratio = target_n_ctx / training_ctx (1.0 when target ≤ training or training
unknown). Implemented as `yarn_coherence_warnings(training_ctx, target_ctx)`
in `yarn.py`, surfaced in `estimate()` warnings and the UI.

## Margin = the honesty knob

`safety_margin_pct` is the user-facing "how scared am I" knob. **Complement
path:** the predictor emits *fully explicit* flags (`-c`, `-ngl`, `-sm`,
`-ts`, `-mg`, `-ctk`, `-ctv`, `--flash-attn`, YaRN flags) **plus**
`--fit on --fit-target <margin MiB>`. Because everything is already explicit,
runtime `--fit` is a no-op safety net — the user gets exactly what the
predictor said. Our number *is* the guarantee, with the margin as the dial.
Never use the word "guarantee" in UI copy; the margin is the honest version.

## Speculative decoding — two branches (this is a bugfix)

The shipped `--spec-type draft` is **invalid**. The real enum (from the
llama.cpp server README) is:

```
none, draft-simple, draft-eagle3, draft-mtp, draft-dflash,
ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache
```

Rewrite the spec surface to this enum, then split the add-on flow:

- **Weightless** (`ngram-*`): ~zero extra VRAM. Auto-offer when headroom
  exists; no "acknowledge the tradeoff" checkbox (there's no VRAM tradeoff).
- **Weighted** (`draft-simple` / `draft-eagle3` / `draft-mtp` /
  `draft-dflash`): a draft model with weights + its own KV. Headroom-gated +
  an acknowledge checkbox. Draft rides the main GPU. Emit the full
  `--spec-draft-*` namespace (`-md`, `--spec-draft-ngl`, `--spec-draft-device`,
  `--spec-draft-n-max`, `--spec-draft-p-min`, `--spec-draft-p-split`), not
  the old `-md`-only stub.

`draft-mtp` (a spec *type*) is distinct from `--mtp N` (MTP head count on
the target). Keep both but emit the right flag for each.

## Backend scope

**Dropped.** Assume it runs. One displayed caveat: "fits in VRAM assuming
your build supports the chosen quant." Keep the existing stock-build
ROCmFP4/ROCmFPX incompatibility warning (that's not a backend issue — no
stock build can read those tensor types at all).

## Quarantined unverified data

Do **not** block v1 on these:

- **nvfp4** (llama.cpp PR #22196): not yet in `QUANT_BPW`. Add a provisional
  row with a `# VERIFY before ship` comment once we have a verified
  bits-per-weight; until then it's absent from the table and can't be
  selected as a floor.
- **dSpark** (#25167 / #25173 / DeepSeek-V4-Flash-DSpark): folds into the
  existing draft-model VRAM path and emits `--spec-type draft-mtp`
  provisionally. For a *predictor*, dSpark's VRAM cost is just
  draft-weights + draft-KV, same as any MTP draft; the spec-type label only
  affects the command string.
- **Vulkan quant matrix:** dropped (consequence of dropping backends).

## UI — two phases, two graphs

1. **Fetch phase:** repo ID + HF token → `parse_hf_range` once → cache
   `ModelArch`. Network happens here, once, never again.
2. **Live phase:** everything is `estimate()` recompute on the cached arch
   (microseconds). Context slider, quant selector, KV-dtype, GPU sizes,
   margin. The recommended command updates live.
3. **Graph 1:** VRAM vs `n_ctx`, one curve per quant — the context ceiling,
   with the slider sitting on it.
4. **Graph 2:** VRAM vs quant at fixed `n_ctx`, **with quality as
   color/rank** — the cost+quality frontier (where "maximize quality at the
   ceiling" becomes visible).

## Implementation order

Done:

1. ✅ **`yarn.py`** — `yarn_coherence_warnings(training_ctx, target_ctx)` +
   `extension_ratio`; wired into `yarn_warnings()` (and thus `estimate()`).
2. ✅ **`report.py`** — emits `--fit on --fit-target <margin MiB>` via
   `_fit_target_mib()` (margin from smallest GPU's VRAM share).
3. ✅ **`draft.py` + `report.py`** — spec surface rewritten to the real
   llama.cpp enum (`draft-simple`, `draft-eagle3`, `draft-mtp`,
   `draft-dflash`, `ngram-*`); weightless (ngram, zero VRAM) vs weighted
   (draft, weights+KV) split; `command_preview` emits `-md` + the full
   `--spec-draft-*` namespace for weighted, only `--spec-type` for
   weightless.
4. ✅ **`quant.py`** — `nvfp4` documented as a commented placeholder with a
   VERIFY note; not selectable as a floor until a verified bpw is sourced.
5. ✅ **Tests** — 67 passing (real-enum spec tests, YaRN coherence-ladder
   tests, `--fit`/`--fit-target` tests, weightless/weighted branch tests).
6. ✅ **`app.py`** — two-phase UI (fetch once → live `estimate()` recompute
   on every change), two graphs (VRAM-vs-n_ctx by quant; VRAM-vs-quant with
   quality color), the "assumes your build supports the chosen quant"
   caveat, and the sub-floor "you asked for it" warning. Syntax-checked;
   graph math verified against presets (gradio not installable locally).

## Out of scope for v1

Per-backend quant compatibility matrices, MLX, Vulkan-specific quants,
runtime auto-tuning (that's llama.cpp's `--fit`), measured perplexity/WikiText
quality tables, throughput/token-per-second modeling.