---
license: other
license_name: qwen-community-1.0
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
base_model: primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8
tags:
- qwen3.8-flash-next
- ple
- nvfp4
---

# Qwen3.8-Flash-Next-mixed-NVFP4-FP8 + NVFP4 group-16 e2m1 PLE table

One repo id for a configuration that otherwise takes two. Contents:

* [`primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8`](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8) at `fd7dbb007b83d4ac41db93ccd1cf2b352c6e8bd2`, **minus its 43
  `ple-bf16-*.safetensors` shards**, with `model.safetensors.index.json` trimmed
  to match (296630 -> 296502 tensors).
* `ples_nvfp4/` -- the NVFP4 group-16 e2m1 quantised n-gram table (26.8 GiB) from
  [`primitive-ai/Qwen3.8-Flash-Next-PLE-quant`](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant) at `4fb644639dc2a5abcf34100c0a3c45fd2a00ba29`.
* The three overlay files from that same repo: `worker_image_quant.py`,
  `ple_layer_quant.py`, `connector_mrv2.py`.

Nothing is re-quantised or otherwise modified; every file is a byte-identical
copy of one of the two sources.

## Why drop the BF16 table

The 43 dropped shards hold exactly 128 tensors, all of them
`...ple.ple_embedding.ngram_embedding.shard_N.weight` -- the 95.4 GiB BF16
n-gram table. Served with the quantised sidecar in `ples_nvfp4/`, nothing reads it:
GPU workers skip PLE layer init under `VLLM_PLE_CPU_OFFLOAD`, and the offload
worker filters those tensors out of the checkpoint stream once
`VLLM_PLE_QUANT_DIR` is set. vLLM still fetches every `*.safetensors` in a repo
before it loads anything, so carrying them means downloading and storing 95.4 GiB
in order to ignore it.

**Use the upstream repos instead if you want the BF16 table** -- for the in-RAM
path (~100 GB of host RAM) or the BF16-on-NVMe overlay. This packaging only makes
sense with the quantised table.

## Serve

Mount the three overlays over the image's site-packages and point
`VLLM_PLE_QUANT_DIR` at `ples_nvfp4/` inside the snapshot. See the
[PLE-quant card](https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-PLE-quant) for the full command and for
primitive-ai's measurements.

`VLLM_GDN_DECODE_KERNEL=triton` is required by this build's FP8 GDN
projections; the default cuda kernel stalls under concurrency.

All credit for both the checkpoint and the tables to primitive-ai; this is a
repackaging. Weights under Qwen Community 1.0, tables under Apache-2.0.