jagat-primitive-org commited on
Commit
4f861b6
·
verified ·
1 Parent(s): 0a85198

Card: the sidecar now also serves FP8-table checkpoints; nvidia's NVFP4 booted on one card with it

Browse files
Files changed (1) hide show
  1. README.md +10 -4
README.md CHANGED
@@ -16,10 +16,16 @@ the CPU-offload worker holds it in BF16 (95.4 GB). This repo ships the same tabl
16
  serves them memory-mapped straight from disk. Host RAM cost becomes page cache only,
17
  reclaimable under pressure.
18
 
19
- Built from the original BF16 tables, so it works with any checkpoint of this model that keeps
20
- them: [our mixed NVFP4/FP8 build](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8),
21
- [our plain NVFP4](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-NVFP4), or the original
22
- model. It does not apply to checkpoints that re-quantized the tables themselves.
 
 
 
 
 
 
23
 
24
  ## Serve
25
 
 
16
  serves them memory-mapped straight from disk. Host RAM cost becomes page cache only,
17
  reclaimable under pressure.
18
 
19
+ Built from the original BF16 tables, so it works with any checkpoint of this model:
20
+ [our mixed NVFP4/FP8 build](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8),
21
+ [our plain NVFP4](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-NVFP4), the original model,
22
+ and, since 2026-09-06, checkpoints that ship their own table in FP8. The overlay stubs the table
23
+ parameter and serves gathers from the sidecar, so the checkpoint's own table tensors are dropped
24
+ before they load. That turns the table format into a non-issue:
25
+ [nvidia's NVFP4 build](https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4), whose card documents
26
+ an 8-GPU deployment because its FP8 table cannot use vLLM's BF16-only host offload, boots on one
27
+ 96 GB card this way (234 s, 88,828 MiB, 348k-token KV pool) and scores 92.1 knowledge / 78.3
28
+ tool-calling under our protocol. Its 53.7 GB table-and-MTP shard can be skipped at download time.
29
 
30
  ## Serve
31