--- license: apache-2.0 base_model: cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit base_model_relation: quantized library_name: paiton pipeline_tag: text-generation inference: false tags: - paiton - amd - rocm - rdna4 - gfx1201 - compiled-model - code --- # Qwen3-Coder 30B with Paiton on R9700 A ready-to-run coding release for one **AMD Radeon AI PRO R9700, 32 GB**. Use terminal chat or connect a coding client to the local OpenAI-compatible API. This Hub repo hosts the **compiled Paiton artifact, manifest, launcher and release information**. It uses the same executable artifact as the published v1.0.0 container. The container fetches the pinned 18.1 GB quantized checkpoint from [cyankiwi](https://huggingface.co/cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit/tree/4bd30395b72ea6045edd04806c4fea448d4467b3) and caches it locally. No model weights or private compiler source are hosted here. This is a Paiton runtime package, not a Transformers checkpoint that can be loaded by passing this repository name to `from_pretrained`. **Version:** v1.0.0 · **GPU:** gfx1201 · **Context:** 4,096 tokens including output · **Concurrent sequences:** 1 or 2 · **Precision:** INT4 G32 / BF16 activations ## Download from Hugging Face and start coding With Linux, Docker, a working AMD GPU driver and the Hugging Face CLI: ```bash hf download EliovpAI/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit-Paiton-RDNA4 \ --revision v1.0.0 --local-dir ./paiton-qwen3-coder bash ./paiton-qwen3-coder/serve-docker.sh --chat ``` Install the CLI with `python3 -m pip install huggingface_hub` if needed. The launcher pulls the prebuilt image, starts the server and opens chat after it is ready. No local compiler or container build is needed. The default cache is `~/.cache/paiton`; set `PAITON_CACHE_DIR` to choose another directory. Use `/reset` for a new conversation and `/quit` to exit chat. The API stays running; stop it with `docker stop paiton-qwen3-coder`. For a download without the HF CLI, use the [ready-to-run archive](https://github.com/Eliovp-BV/paiton-vllm-plugin/releases/download/qwen3-coder-30b-awq-rdna4-v1.0.0/paiton-qwen3-coder-r9700-v1.0.0.tar.gz) and run `./run.sh --chat` after extracting it. ## Coding clients and API | Client setting | Value | | --- | --- | | Provider | OpenAI-compatible | | Base URL | `http://127.0.0.1:8010/v1` | | Model | `qwen3-coder` | | API key if the client requires a value | `local` | | Context budget | `4096`, including output | | Suggested maximum output | `1024` | The endpoint is bound to localhost and needs no authentication. Ordinary chat, streaming and automatic function tool calls are enabled. Both ordinary and streamed tool-call arguments were checked with the published runtime. Use focused functions, files and coding questions; whole-repository agent workloads and long-context quality are not qualified. ```bash curl http://127.0.0.1:8010/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"qwen3-coder","messages":[{"role":"user","content":"Write a Python binary search with tests."}],"max_tokens":1024,"temperature":0}' ``` Omit `--chat` to start only the API. `--offline` requires the pinned checkpoint in the cache. `--download-only` populates the cache without loading the GPU. Use `--stock` for the stock comparison; stop the existing server before changing modes. See the [complete guide](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/main/models/Qwen3-Coder-30B/README.md) for port, container-name and cache options. ## Hardware, performance and quality The tested machine had an R9700, 16 GB host RAM with 4 GB swap, an i5-8400 and AMD driver 6.19.14.31400000. Reserve at least **30 GB free disk** for weights, the roughly 5.1 GB image and caches. Model loading took about 46 seconds; cached startup takes roughly two minutes, with extra time for fresh compilation caches. The first weight download is separate. Sampled warm GPU usage peaked near **20.1 GiB**, with the weights fully resident on the GPU. | Matched concurrency | Stock output tokens/s | Paiton output tokens/s | Gain | | --- | ---: | ---: | ---: | | 1 | 104.42 | 126.68 | 21.3% | | 2 | 101.61 | 172.82 | 70.1% | These are the already-published fixed short-workload results: two measured runs of 16 requests per engine/setting, 229–241 actual input tokens and exactly 256 output tokens. All 128 measured requests completed with matching counts. Mean request latency fell from 2.451 to 2.020 seconds at concurrency 1 and from 5.038 to 2.961 seconds at concurrency 2. This comparison uses the fastest stock configuration tested, not every possible checkpoint or runtime. Both engines passed four executable coding checks and scored 7/8 on the small quality suite. Both failed the JSON sorting check by returning strings instead of numbers. Numerical and generated-text differences remain; the suite does not establish comprehensive quality parity. No long-context, higher-concurrency, LoRA or speculative-decoding performance is qualified. No new performance claim is introduced by publishing these files on Hugging Face. [Full measurements, latency definitions and limitations](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/9148505697c70ed3e0a68f298ca6860579db0fbb/models/Qwen3-Coder-30B/BENCHMARKS.md) · [Reproduction commands](https://github.com/Eliovp-BV/paiton-vllm-plugin/blob/main/models/Qwen3-Coder-30B/REPRODUCE.md) ## Artifact use and provenance The normal launcher uses the identical artifact already inside the container. For manual use of the downloaded Hub artifact, start the image with this read-only mount and artifact path: ```bash docker run -d --name paiton-qwen3-coder \ --device /dev/kfd --device /dev/dri --group-add video --ipc=host \ -p 127.0.0.1:8010:8010 -v paiton-qwen3-coder-cache:/models/cache \ -v "$PWD/paiton-qwen3-coder/overlay:/models/paiton-overlay:ro" \ -e PAITON_QWEN3_CODER_ARTIFACT=/models/paiton-overlay/qwen3_coder_moe_w4a16_g32_gfx1201.so \ ghcr.io/eliovp/paiton-vllm-plugin:qwen3-coder-30b-awq-rdna4-v1.0.0 ``` Keep the adjacent `.json` manifest with the `.so`. The runtime checks the artifact hash and GPU architecture before loading it. The immutable image is: `ghcr.io/eliovp/paiton-vllm-plugin@sha256:fb47f4ab6073da943e553849985c81326e33b425e1017f97f3f79fad09586a8e` The checkpoint is pinned to `cyankiwi/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit` revision `4bd30395b72ea6045edd04806c4fea448d4467b3`, with compressed-tensors symmetric INT4 groups of 32 and BF16 activations. It derives from the official [Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct). The quantizer does not identify its exact upstream weight revision. The Hub quantization relationship associates this runtime package with that checkpoint; this release performs no additional weight quantization. [SHA256SUMS](SHA256SUMS) covers the package files and [paiton-hub.json](paiton-hub.json) records artifact and runtime provenance. The stack uses HIP 7.14.60850, PyTorch 2.12.0+rocm7.14.0, Transformers 5.15.1, and vLLM commit `39bd959b582c85e78e7e0326d49042ce7c3c07ed`. ## Licenses and links [Apache-2.0](LICENSE), [Qwen license](QWEN_LICENSE), [component notices](THIRD_PARTY_NOTICES.md), [model provenance](NOTICE.md) and [retained license texts](LICENSES/) accompany this package. Original checkpoint terms continue to apply to weights fetched from the publisher. [Public runtime](https://github.com/Eliovp-BV/paiton-vllm-plugin) · [Release files](https://github.com/Eliovp-BV/paiton-vllm-plugin/releases/tag/qwen3-coder-30b-awq-rdna4-v1.0.0) · [Paiton](https://eliovp.com/products/paiton)