bw24-bench — benchmark artifacts for bw24

Reproduction artifacts for the performance claims in github.com/avifenesh/bw24 — a from-scratch LLM inference engine for RTX 5090 Laptop (sm_120a). If a claim depends on a specific artifact, the artifact is public here.

Drafts (drafts/<model>/) — the standard regime, one file per model

Every bw24 spec board row runs one trimmed draft file built by the documented regime (docs/DRAFT-REGIME.md): FR-Spec ranks derived from the model's own generations (never transferred between models — measured −12 acceptance pts), MTP block extracted byte-verbatim from the published model GGUF, head requantized NVFP4 after trimming (zero measured acceptance cost), block Q4_K_M (faster AND higher acceptance than Q8_0). Serve with BW24_MTP_DRAFT=<draft.gguf> — no other flags.

Directory	Source model (exact bytes)	Board row (e2e tok/s p1/p2/p3, vs llama.cpp same-window)
drafts/qwen35-9b-nvfp4/	Qwen3.5-9B NVFP4 MTP GGUF	281 / 212 / 187 = 2.30x / 1.74x / 1.59x
drafts/qwen36-27b-nvfp4/	nvidia/Qwen3.6-27B-NVFP4 → Q4_K_M GGUF	116 / 101 / 86 = 1.27x / 1.08x / 1.06x
drafts/qwen36-35b-a3b-iq4xs/	unsloth Qwen3.6-35B-A3B UD-IQ4_XS	281 / 260 / 258 = 1.19x / 1.49x / 1.49x
drafts/qwen36-27b-unsloth-nvfp4/	unsloth/Qwen3.6-27B-NVFP4 → GGUF (recipe in bw24 jsonl)	plain parity with the nvidia daily; spec p3 win / p2 loss (jsonl 2026-07-17)

Each directory carries the draft (draft-owntrim-nvfp4head-q4blk.gguf) and the rank file it was built from (owngen-ranks-32768.txt, one token id per line, rank order — derived 2026-07-17/18, 218-prompt mixed corpus, ~110k own-generated tokens per model).

Use ours for these exact models. For any other model, requant, or finetune, build your own in two commands (a finetune's distribution moved, so its draft must too):

./target/release/frspec-owngen model.gguf ranks.gguf 32768        # ranks from the model's OWN generations
tools/make-trimmed-draft.sh model.gguf ranks.gguf.txt draft.gguf  # extract + trim + quantize

Gemma ranks (drafts/gemma4-*/)

Gemma drafters ship as standalone MTP/assistant GGUFs (byte-verbatim by provenance — law 2 built in); the FR-Spec trim applies at load: BW24_GEMMA_DRAFT_RANKS=<ranks.txt>. Own-gen rank files per model (law 1):

Directory	Model	Serving note
drafts/gemma4-26b-a4b-qat/	Gemma-4 26B-A4B QAT Q4_0	trim adopted (e2e-neutral vs corpus ranks, correct provenance)
drafts/gemma4-31b-qat/	Gemma-4 31B QAT Q4_0	trim adopted on BOTH cells via the serve-time ADAPTIVE trim (BW24_GEMMA_TRIM_ADAPT, on by default with ranks): coverage escapes learned live from prompt+verify tokens, persisted to <ranks>.learned — flipped the chat cell from −17% to +2.5% (bw24 jsonl 2026-07-19)
drafts/gemma4-e4b-qat/	Gemma-4 E4B QAT Q4_0	ranks published; serving stays UNTRIMMED (measured e2e wash — small head)

Legacy trims (top level, Qwen3.6-27B)

The pre-regime artifacts the earlier boards used — superseded by drafts/ but kept because published claims referenced them:

File	Ranking	Status
mtp-Qwen3.6-27B-Q4_K_M-frspec32768.gguf	generic corpus-frequency	superseded by drafts/qwen36-27b-nvfp4/
mtp-Qwen3.6-27B-Q4_K_M-frspec-code75-32768.gguf	code-skewed corpus	superseded
mtp-Qwen3.6-27B-Q4_K_M-frspec-balanced32768.gguf	balanced corpus	superseded

Prompts + configs

prompts/ holds the exact p1/p2/p3 board prompts; CONFIGS.md the per-cell engine configs and llama.cpp pairing flags. Protocol: interleaved same-window N≥2, power state pinned, one engine on the GPU at a time — research/benchmarks.md.