Update README.md
Browse files
README.md
CHANGED
|
@@ -18,6 +18,8 @@ Check the original model card for information about this model.
|
|
| 18 |
|
| 19 |
## ⚠️ Known vLLM Compatibility Issues
|
| 20 |
|
|
|
|
|
|
|
| 21 |
This model currently does not work with vLLM due to CUTLASS FP4 kernel constraints - the 103-expert gate layer fails with `Expected n to be divisible by 32`. See [#24921](https://github.com/vllm-project/vllm/issues/24921), [#30934](https://github.com/vllm-project/vllm/issues/30934)
|
| 22 |
|
| 23 |
Additionally, Blackwell consumer/workstation GPUs (RTX 5090, RTX PRO 6000, DGX Spark) have a separate issue where FlashInfer FP4 GEMM doesn't support SM120/SM121. See [#31074](https://github.com/vllm-project/vllm/issues/31074), [#30163](https://github.com/vllm-project/vllm/issues/30163), [#23497](https://github.com/vllm-project/vllm/issues/23497)
|
|
|
|
| 18 |
|
| 19 |
## ⚠️ Known vLLM Compatibility Issues
|
| 20 |
|
| 21 |
+
[This discussion](https://huggingface.co/Firworks/Qwen3-Coder-REAP-25B-A3B-nvfp4/discussions/1) contains a workaround to get this model running in VLLM until the below issues have been fixed/merged in VLLM.
|
| 22 |
+
|
| 23 |
This model currently does not work with vLLM due to CUTLASS FP4 kernel constraints - the 103-expert gate layer fails with `Expected n to be divisible by 32`. See [#24921](https://github.com/vllm-project/vllm/issues/24921), [#30934](https://github.com/vllm-project/vllm/issues/30934)
|
| 24 |
|
| 25 |
Additionally, Blackwell consumer/workstation GPUs (RTX 5090, RTX PRO 6000, DGX Spark) have a separate issue where FlashInfer FP4 GEMM doesn't support SM120/SM121. See [#31074](https://github.com/vllm-project/vllm/issues/31074), [#30163](https://github.com/vllm-project/vllm/issues/30163), [#23497](https://github.com/vllm-project/vllm/issues/23497)
|