--- license: apache-2.0 base_model: openai/gpt-oss-20b library_name: paiton pipeline_tag: text-generation inference: false tags: - paiton - amd - rocm - rdna4 - gfx1201 - compiled-model - mxfp4 --- # GPT-OSS-20B on AMD RDNA4 — 2.18× speedup with Paiton **54% lower end-to-end latency on one Radeon AI PRO R9700**, using OpenAI's original MXFP4 checkpoint. The stock reference already uses **vLLM O2 and full decode graph capture**. No new weight quantization or speculative decoding. | Primary scenario: 512 input / 256 output tokens, concurrency 1 | Median latency | |---|---:| | Fastest tested stock vLLM reference | 4.819 s | | Paiton | **2.214 s** | 32 measured requests per mode after warmup; **2.176× speedup / 54.047% lower latency**. The corrected-image stock run measured 6.110 s and the original-runtime repeat 6.265 s. Their unexplained timing shift is retained; the headline uses the faster 4.819-second reference. These results do not establish a universal speedup or that every possible stock setting has been exhausted. This free community repository contains **compiled artifacts and manifests**, not model weights or private compiler source. The Docker image already bundles these artifacts and downloads weights directly from OpenAI at the pinned revision. **Release:** v1.0.0 · **GPU:** R9700, gfx1201, 64 CU / 32 WGP, 32 GB VRAM · **Context:** 8,192 tokens including output · **Scheduled requests:** up to 2. ## One-command Docker launch Requires Linux x86-64, Docker and working AMD GPU access through `/dev/kfd` and `/dev/dri`. The image includes the qualified ROCm 7.14 and vLLM runtime. ```bash docker run -d --name paiton-gptoss \ --device /dev/kfd --device /dev/dri --group-add video --ipc=host \ -p 8020:8020 -v paiton-gptoss-cache:/models/cache \ ghcr.io/eliovp/paiton-vllm-plugin:gpt-oss-20b-mxfp4-rdna4-v1.0.0 ``` The API listens on `0.0.0.0:8020`. Model and compilation caches persist in the named volume. Follow `docker logs -f paiton-gptoss` and wait for readiness. The public checkpoint does not require a Hugging Face token. Harmony's pinned vocabulary is embedded for offline chat with cached model weights. ```bash curl --fail http://localhost:8020/v1/models docker exec -it paiton-gptoss python3 /opt/paiton/gpt-oss/chat.py curl --fail http://localhost:8020/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"gpt-oss-20b","messages":[{"role":"user","content":"Explain binary search in two sentences."}],"reasoning_effort":"low","max_tokens":256,"temperature":0}' ``` To compare stock, stop this server and run the same Docker command with `--stock` after the image name. The checkpoint and serving settings remain matched. For an immutable image reference, use: ```text ghcr.io/eliovp/paiton-vllm-plugin@sha256:0cf9c11304ade58e91d97db876a4462781fa2c18d2d4846f26d56fe5d207669d ``` ## What Paiton optimizes GPT-OSS has approximately 21B total / 3.6B active parameters. All experts stay on the GPU. Paiton replaces supported one/two-token expert computation with compiled RDNA4 kernels. Bulk prefill and unsupported shapes use stock experts. Attention, sinks, sliding windows, top-4 routing, sampling and Harmony parsing remain in vLLM. Original MXFP4 storage is unpacked for FP32 accumulation, retaining the specified BF16 activation and weighted-partial boundaries. This is not native FP4 arithmetic. KV cache remains BF16. There is no requantization of the released checkpoint. Pinned model revision: `6cee5e81ee83917806bbde320786a8fb61efebee`. The [upstream model](https://huggingface.co/openai/gpt-oss-20b) supplies the weights. ## Quality, memory and limits - Stock and Paiton both pass **18/20 fixed quality tasks**, with matching prompt token IDs and no task-level regressions in that set. Both retain a malformed coding-channel failure and an unconstrained JSON-fence failure. - All **24 numerical cases** and **10 API checks per mode** pass, including streaming, tool continuation, reasoning efforts and concurrent long retrieval. This small suite does not prove general quality parity. - Tested with 8,192 total context tokens, two scheduled requests and 2 GiB BF16 KV. Sampled driver memory peaks around **17 GiB**; all experts are resident. - Qualified host: 15.52 GiB RAM and 4 GiB swap. Reserve roughly 70 GB disk for the container, extracted runtime, 13.8 GB checkpoint tensors and caches. - One cold compilation-cache startup with cached weights took about 155 s stock and 142 s Paiton. Downloads are additional; these are single observations. - No LoRA, speculative decoding or multi-GPU qualification. Reasoning consumes the output budget; a small budget can end before a visible answer. The primary throughput counts reasoning explicitly; it is not useful-answer throughput. Natural-stop chat and channel counts are reported separately. The ROCm kernel profiler failed, so invalid kernel traces are excluded; engine prefill/decode timings and end-to-end serving measurements are retained. ## Download the compiled artifacts The container includes the same overlay. To download it separately: ```bash hf download EliovpAI/GPT-OSS-20B-MXFP4-Paiton-RDNA4 --revision v1.0.0 \ --include 'overlay/*' --local-dir ./paiton-gptoss-hf ``` For manual use, add these options before the Docker image name: ```bash -v "$PWD/paiton-gptoss-hf/overlay:/models/paiton-overlay:ro" \ -e PAITON_GPTOSS_ARTIFACT=/models/paiton-overlay/gptoss_mxfp4_decode_gfx1201.so ``` Keep the `.so` and `.json` together. The loader checks architecture, ABI and SHA-256. This is a compiled runtime overlay, not a standalone Transformers checkpoint. ## Reproduce and inspect - [Model guide](https://github.com/Eliovp-BV/paiton-vllm-plugin/tree/9467d7e4982a0a9d8b1f5176571e7800a48f58cb/models/GPT-OSS-20B) - [Full benchmark matrix, variability and limitations](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/9467d7e4982a0a9d8b1f5176571e7800a48f58cb/models/GPT-OSS-20B/BENCHMARKS.md) - [Reproduction and fixed evaluation set](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/9467d7e4982a0a9d8b1f5176571e7800a48f58cb/models/GPT-OSS-20B/REPRODUCE.md) - [Machine-readable results](results.json), [image pins](container-images.json), [file checksums](SHA256SUMS) and [artifact provenance](paiton-hub.json) - [Paiton community models](https://github.com/Eliovp-BV/paiton-vllm-plugin) OpenAI's checkpoint is Apache-2.0; its license and usage policy are included. Paiton integration and compiled-runtime notices are in `LICENSE`, `LICENSES/`, `NOTICE.md` and `THIRD_PARTY_NOTICES.md`. The MIT tiktoken notice is included.