Optimized vLLM and Spark Arena results
The selected configuration is the non-speculative T5 recipe in
../recipes/vllm-optimized-sm121-tp2.yaml.
It keeps the T3 FULL_DECODE_ONLY CUDA Graph configuration and extends the
qualified operating envelope to a 262,144-token API ceiling with 12 GiB of
explicit KV-cache reservation. Growing-prefix and isolated long-context gates
passed 12/12 through 250,000 tokens.
Quality and fixed-workload selection
| Comparison | Quality delta | Throughput delta |
|---|---|---|
| Upstream -> NVFP4 S0, GSM8K | +0.0 pt | +0.725 tok/s |
| NVFP4 S0 -> selected T3, GSM8K | +0.0 pt | +6.419 tok/s |
| NVFP4 S0 -> selected T3, fixed workload | n/a | +9.376 tok/s |
| T3 -> selected long-context T5, GSM8K | -1.0 pt | -0.405 tok/s |
| T3 -> selected long-context T5, fixed workload | n/a | -0.042 tok/s |
T6 DSpark k=3 and T7 DSpark k=5 remained inside the three-point quality gate, but reduced fixed-workload throughput by 7.349 and 9.039 tok/s respectively. They were rejected; T5 remains selected.
Spark Arena submission
- Submission:
sub1788319379140 - Status: completed
- Profile:
official/spark-arena-v2 - Runtime: vLLM, tensor parallel 2, two DGX Spark nodes
- Cells: 28/28 completed, 0 failed, three runs per cell
- Model identity:
s-zaizen/DeepSeek-V4-Flash-Vision-Exp-NVFP4
Representative results:
| Depth | Concurrency | PP tok/s | TG tok/s | TTFR ms |
|---|---|---|---|---|
| 0 | 1 | 1206.7 | 25.4 | 1699.1 |
| 0 | 2 | 483.6 | 21.5 | 5087.1 |
| 0 | 10 | 325.5 | 19.2 | 32509.2 |
| 16,384 | 1 | 130.2 | 25.2 | 15731.4 |
| 16,384 | 2 | 112.0 | 9.8 | 26150.1 |
| 16,384 | 10 | 100.9 | 6.6 | 109304.5 |
| 100,000 | 1 | 22.0 | 24.9 | 93032.5 |
| 100,000 | 2 | 21.4 | 2.4 | 142417.9 |
| 100,000 | 10 | 21.0 | 1.4 | 534753.7 |
The accompanying CSV and JSON contain all cells. Their model values were
mechanically replaced with the canonical Hugging Face ID before publication;
all numeric benchmark values remain byte-for-byte equal to the original fields.