# Ornith-1.0-35B GGUF Long-Context TTFT

Date: 2026-06-28

How prefill latency (time-to-first-token) scales with prompt/context length on a
single GPU, single stream. This complements the short-context throughput sweep in
[llamacpp-quant-benchmarks.md](llamacpp-quant-benchmarks.md) (which fixes
`n_ctx=512`/slot and uses ~23-token prompts).

## Config

- Backend: local llama.cpp CUDA server, commit `050ee92d04c2e1f639025786dea701c70e7d4204`.
- GPU/driver: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition, driver 580.159.03.
- Mode: single GPU, `tp=1`, `CTX_SIZE=131072`, `PARALLEL=1`, `n_ctx/slot=131072`,
  `REASONING=off`, `CACHE_RAM=0`.
- Method: per context length, prompts truncated to the exact token count, 64-token
  generations, deterministic; one warm-up discarded, then ≥20 requests measured.
- Validation: all 3 quants have 140 rows, 0 failed requests, exact prompt token
  counts, no skipped contexts.

## Results — p95 / p50 TTFT (ms) and decode tok/s

| Context tokens | Q4_K_M p50 | Q4_K_M p95 | Q4_K_M tok/s | IQ4_XS p50 | IQ4_XS p95 | IQ4_XS tok/s | MTP-graft p50 | MTP-graft p95 | MTP-graft tok/s |
|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 512 | 91.8 | 94.1 | 188.3 | 87.1 | 88.7 | 177.4 | 86.0 | 87.8 | 184.0 |
| 1024 | 169.6 | 172.2 | 145.8 | 159.9 | 161.6 | 139.9 | 157.6 | 159.0 | 155.9 |
| 2048 | 341.8 | 346.1 | 97.9 | 316.1 | 318.7 | 112.0 | 307.6 | 308.9 | 113.9 |
| 4096 | 699.2 | 702.2 | 67.2 | 647.0 | 653.4 | 70.7 | 625.7 | 631.3 | 69.0 |
| 8192 | 1447.5 | 1458.0 | 33.7 | 1342.9 | 1348.5 | 39.8 | 1297.4 | 1304.0 | 40.9 |
| 16384 | 3009.7 | 3030.4 | 19.5 | 2793.2 | 2804.0 | 20.9 | 2709.8 | 2737.3 | 21.5 |
| 32768 | 6302.4 | 6313.0 | 9.7 | 5829.9 | 5853.1 | 10.5 | 5673.5 | 5696.6 | 10.7 |

TTFT grows roughly with context length (prefill cost), reaching ~6.3 s at 32k
tokens for Q4_K_M. The IQ4_XS body and the IQ4_XS-MTP graft prefill slightly
faster than Q4_K_M at every length (e.g. 5696.6 vs 6313.0 ms p95 at 32k), and
decode tok/s falls from ~180–190 at 512 tokens to ~10 at 32k as the KV cache grows.

## Raw artifacts

Single-stream (`PARALLEL=1`) long-context runs:

- `benchmarks/raw/llamacpp-q4-k-m-longctx-c1.jsonl`
- `benchmarks/raw/llamacpp-iq4-xs-longctx-c1.jsonl`
- `benchmarks/raw/llamacpp-iq4-xs-mtp-graft-headq6-longctx-c1.jsonl`

Each row carries `prompt_tokens` and `ttft_s` in the same schema as the other
`benchmarks/raw/*.jsonl` files.