harness: protoLab evals @ claw-eval 49d2b6a

judge: protolabs/reasoning

kind: gate

grading: solver


grading: deterministic


grading: judge


grading: execution


kind: coherence

grading: detectors+adversarial-judge


kind: speed

config: vllm-0.22.1 moe-backend=marlin

hardware: RTX PRO 6000 Blackwell sm120

method: vllm bench serve seeded random


grading: per-suite

note: identical to FP8


note: 16K thinking budget; verbose-build caveat


note: paired-84; zh cluster softer


hardware: RTX PRO 6000 Blackwell

config: llama.cpp draft-mtp

note: mean n=6 prompts (287-336)


hardware: RTX A6000 Ampere

note: +46% over no-spec 127.7; accept 0.62


config: llama.cpp no-spec