Card: field report (64 GB host) — pin --max-model-len/--max-num-seqs in the serve blocks, vm.overcommit_memory note, MTP+xgrammar caveat

README.md

@@ -39,13 +39,16 @@ docker run --gpus all --ipc=host -p 8000:8000 \
   --model primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8 \
   --distributed-executor-backend mp \
   --gpu-memory-utilization 0.92 \
+  --max-model-len 32768 --max-num-seqs 36 \
   --enable-auto-tool-choice --tool-call-parser qwen3_coder \
   --reasoning-parser qwen3
 ```
 
 Drop `-e VLLM_GDN_DECODE_KERNEL=triton` when serving the plain NVFP4 build. No container
 memory cap needed: unlike the BF16 disk path, the quantized tables fit the page cache next to
-checkpoint streaming.
+checkpoint streaming. Two flags were added to the block on 2026-09-06 after a field report: the image's default `--max-num-seqs` exceeds the Mamba state blocks left at 0.92 utilization and CUDA graph capture aborts, and at the native 262,144 context with MTP the KV cache no longer fits after the draft head. 32,768 / 36 is what the table below was measured with; the report ran 135,168 / 64 with MTP.
+
+On a host with less than about 100 GB of RAM plus swap, set `sysctl vm.overcommit_memory=1` as well. The offload worker constructs the 95 GB BF16 table parameter before this overlay stubs it and points gathers at the sidecar, and the kernel's default heuristic refuses a single allocation larger than RAM + swap (`DefaultCPUAllocator: can't allocate memory`). A revised `ple_layer_quant.py` that builds the parameter on the meta device instead is in test and will replace the sysctl requirement.
 
 ## Measured
 
@@ -65,8 +68,16 @@ the pooled 200-item suite, mean of three runs per format (suite repeat spread ±
 formats sit in one band). Generation-sanity gates passed on every configuration.
 
 Validated end to end inside a **48 GB container** (INT4 table): sanity PASS, tool-calling
-80.5, 79.4 tok/s @ 1 and 486 @ 32 — within noise of uncapped. A 64 GB-RAM host serves this
-180B model.
+80.5, 79.4 tok/s @ 1 and 486 @ 32 — within noise of uncapped. The cap was a cgroup limit on a
+176 GB host; a real 64 GB host additionally needs the `vm.overcommit_memory` setting from the serve section.
+
+**Field report, 64 GB host.** [Marzero](https://github.com/MarcoPizeta/flash-next-rtxpro6000-bench) ran this INT4 table with the mixed checkpoint on one RTX PRO
+6000 and 64 GB of DDR5 (60.9 GiB usable), MTP 3, `--max-model-len 135168 --max-num-seqs 64`: decode with MTP
+102 / 140 / 99 tok/s at 1k / 8k / 32k input single-stream and 296 / 282 / 151 at concurrency 4, prefill about
+28k tok/s from 8k to 128k, host RAM 10 to 12 GB, KV pool 217k tokens with MTP (503k without), 2.1 to 2.3 Wh per
+1k output tokens, all 32k x 16 requests served. Their protocol, their box; the raw JSON and scripts are in the
+repo. One operational note from the same run: the offload worker keeps three cores in a busy loop, which pushed
+an air-cooled Threadripper to 95 °C until clocks were capped at 4.5 GHz.
 
 ## Format