Instructions to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
llama cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
llama cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Use Docker

docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

LM Studio
Jan

vLLM

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Ollama
How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Ollama:
```
ollama run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
```
Unsloth Desktop

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Docker Model Runner:
```
docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
```

Lemonade

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Run and chat with the model

lemonade run user.Qwen3.8-Flash-CIRU-STRIX-IU4-Q8_0

List all available models

lemonade list

Hermes Agent

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-Flash-CIRU-STRIX-IU4

A quality-first, long-context Qwen3.8-Flash-Next build for fast local inference on AMD Strix Halo.

This is a text-only, custom-runtime release. The main GGUF is not standalone: it requires the CIRU llama.cpp runtime and all three files under ple/. The MTP draft under mtp/ enables the headline speculative-decoding profile. Stock llama.cpp and Hugging Face hosted inference cannot run this package correctly.

Use runtime version 1.1. Version 1.1 fixes the version 1.0 cached-prefix/MTP issue, and the normal production prompt cache remains enabled. If you already downloaded the model, PLE, and MTP files, do not download them again; only rebuild the runtime from tag v1.1.

Headline results

Result	Score	Scope
HumanEval	160/164 (97.56%)	Full 164-task local-custom chat run
HumanEval+	155/164 (94.51%)	Full 164-task EvalPlus run
ARC-Challenge	1,143/1,172 (97.53%)	Full EvalScope dataset
ToolEval Standard	115/138 points (83.33%)	69 local-custom cases
ToolEval Hard	23/30 points (76.67%)	15 local-custom hard cases
H121 8K cold prefill	359.43 tok/s	8,192-token prompt
H121 8K generation	30.80 tok/s	128 generated tokens, MTP depth 3
Long-context coverage	131,072 prompt tokens	Cold exact-count context ladder

Quality suites used the same released model artifacts on the earlier H96 depth-1 runtime. Version 1.0 added the H121 allocator-lifetime correction. Version 1.1 adds the cached-prefix/MTP state fix and completed the full 20-scenario HermesAgent cache-on validation without an MTP position or server-runtime error. Local-custom and sampled results are labeled and are not claimed as canonical leaderboard submissions.

Why this build

Qwen3.8-Flash-CIRU-STRIX-IU4 is designed around the Ryzen AI Max+ 395's large unified-memory envelope and gfx1151 matrix instructions:

Quality-first mixed storage. Routed expert weights use Q4_1 while the protected core remains Q5_K, Q5_1, Q8_0, BF16, or F32.
Native IU4 execution. Stored Q4_1 expert weights use a tuned unsigned-IU4 WMMA path on gfx1151. IU4 describes runtime execution, not a uniform four-bit model or custom GGUF storage type.
NVMe-paged exact FP8 PLE. The mandatory CIRUPLE1 sidecar pages exact FP8 E4M3 PLE weights from SSD with a 4 GiB decoded-page cache.
MTP speculation. The included Q8_0 draft supports the public depth-3 speculative profile.
Production caching. Version 1.1 enables prompt reuse, an 8 GiB RAM prompt cache, idle-slot caching, and synchronized target/draft/MTP context checkpoints.
Long context. Native server configuration is 262,144 tokens; cold performance has been measured through 131,072 prompt tokens.

Version 1.1 cached-prefix/MTP fix

Version 1.1 keeps the production prompt cache enabled. It fixes the version 1.0 multi-turn failure that could occur when a cached target prefix was reused while MTP's hidden-state timeline remained at a different position.

The runtime now saves and restores MTP pending hidden state together with compatible target and draft context checkpoints. If no compatible checkpoint exists, it safely resets that request's cached MTP state instead of continuing with mismatched positions. The model, PLE, and MTP files are unchanged.

Validation covered direct branched-prefix reuse, the previously failing HA-07 cached multi-turn case, HA-01, HA-20, and the full 20-scenario HermesAgent run with production cache settings. The full run completed all 20 scenarios with no decreasing-position, HSA, pager, or server-runtime error.

Version 1.0 allocator-lifetime fix

During sustained MTP depth-3 generation, the pre-release runtime could intermittently page fault in Q8_0 GET_ROWS. The root cause was allocator lifetime: reused Qwen4Exp M=1 token and hidden-state inputs were marked input-only, so their graph allocations could be recycled between continuation steps.

H121 retains both tensors as graph outputs:

ggml_set_output(inp->tokens);
ggml_set_output(inp->h);

This adds no kernel, copy, synchronization, fallback, or model change. The corrected runtime completed a 6,009-token coding-generation stress test with no HSA, pager, nonfinite, or server failure, then reproduced the 8K+128 row at 359.4267 prefill tok/s and 30.7970 generation tok/s. Version 1.0 is preserved at tag v1.0; the earlier v1.0.0-h121 tag remains available for compatibility.

Download

The complete package is 135,962,881,135 bytes (126.625 GiB):

python -m pip install -U "huggingface_hub[cli]"
hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 \
  --local-dir ./Qwen3.8-Flash-CIRU-STRIX-IU4
cd Qwen3.8-Flash-CIRU-STRIX-IU4
sha256sum -c checksums.sha256

Expected layout:

Qwen3.8-Flash-CIRU-STRIX-IU4/
├── Qwen3.8-Flash-CIRU-STRIX-IU4.gguf
├── mtp/
│   └── Qwen3.8-Flash-CIRU-STRIX-IU4-MTP-Q8_0.gguf
└── ple/
    ├── ple.manifest.json
    ├── ple.payload.bin
    └── ple.scale.bf16

File	Bytes	GiB	SHA-256
`Qwen3.8-Flash-CIRU-STRIX-IU4.gguf`	79,397,818,720	73.944981	`c0ea11e4e24d0f909720b6c4e7462aa1e6fbf5e0f6acc796063f2aed4cf46ed0`
`mtp/Qwen3.8-Flash-CIRU-STRIX-IU4-MTP-Q8_0.gguf`	4,135,893,248	3.851851	`e6743badef1f2619fcb5addfa4344a2a3368cb75214735117e3af80c70b80642`
`ple/ple.payload.bin`	52,429,053,952	48.828362	`687fc742efb6888c6cd7cf9c80cb4b1ac8cb4707b9409c206699c43363e239b2`
`ple/ple.manifest.json`	115,213	0.000107	`eb7404ce5ef056729452df10ee888e0c300cd0459121444be3313c51788cc171`
`ple/ple.scale.bf16`	2	<0.000001	`c7c58bd6007672362da2106fdbfaf9f50629e4bdf8598169c598027394ef9791`

The target and ple/ are mandatory. The MTP file may be omitted only when speculative decoding is disabled.

Build the runtime

The measured path is Linux x86-64, AMD ROCm/TheRock 10-class toolchain, gfx1151, on a Ryzen AI Max+ 395 / Radeon 8060S.

git clone --branch v1.1 \
  https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4.git runtime
cd runtime
ROCM_ROOT=/opt/rocm ./scripts/ciru/build-linux-amd.sh

Cross-platform documentation:

Ubuntu/Debian, Fedora/RHEL, Arch, and WSL2/ROCDXG are source-compatible but unvalidated for this release. Native Windows CPU and macOS Metal are portability paths and do not include the validated Strix HIP or Linux P16/O_DIRECT performance path. On WSL2, keep ple/ple.payload.bin on the Linux ext4 filesystem, not /mnt/c.

Production setup for version 1.1

From the version 1.1 runtime checkout:

MODEL_DIR=/absolute/path/to/Qwen3.8-Flash-CIRU-STRIX-IU4 \
  ./scripts/ciru/run-server.sh

Prompt caching and MTP are both enabled in version 1.1. A normal API request can keep prompt caching on:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.8-Flash-CIRU-STRIX-IU4",
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 1.0,
    "top_p": 0.95,
    "top_k": 20,
    "cache_prompt": true
  }'

The version 1.1 production configuration uses:

PLE cache: 4 GiB, 16 pager workers.
Prompt cache: enabled, 8 GiB RAM cache, idle-slot caching.
Context checkpoints: 32, minimum spacing 8,192 tokens.
Context/batch/microbatch: 262,144 / 2,048 / 512.
One slot, F16 target KV, Q8_0 draft KV, flash attention on.
MTP depth 3 with p_split=0.10.
Loopback-only bind at 127.0.0.1:8080.

The 4 GiB PLE cache is separate from the 8 GiB prompt cache: the PLE cache stores decoded model-weight pages from NVMe, while the prompt cache reuses conversation prefixes.

Expanded settings, no-MTP mode, and network hardening are documented in RUNNING.md.

Recommended sampling

Upstream Qwen recommendations:

Mode	Temperature	Top-p	Top-k	Min-p	Presence penalty	Repetition penalty
Thinking, default	1.0	0.95	20	0	0	1.0
Non-thinking	0.7	0.80	20	0	1.5	1.0

For non-thinking mode, send "chat_template_kwargs":{"enable_thinking":false}. Keep "cache_prompt":true for normal reusable chat traffic on version 1.1.

Quality results

Benchmark	Result	Coverage and caveat
HumanEval	160/164 (97.561%)	Full; local-custom chat, first sample, no retries/repairs
HumanEval+	155/164 (94.512%)	Full; EvalPlus 0.1.10
ARC-Challenge	1,143/1,172 (97.526%)	Full EvalScope dataset
ToolEval Standard	115/138 (83.33%)	69 local-custom cases; 0 runtime/transport errors
ToolEval Hard	23/30 (76.67%)	15 local-custom hard cases; 0 runtime/transport errors
GPQA-Diamond	46/50 (92.0%)	Sampled first/default-order 50 of 198; not a full score
MMLU-Pro	61/70 (87.143%)	Sampled 5 from each of 14 subjects
GSM8K	97/100 (97.0%)	Sampled local run
IFEval strict	92/100 (92.0%)	Sampled prompt-level-strict local run

HumanEval serving averages were 211.108 prefill tok/s and 31.531 generation tok/s, with depth-1 MTP acceptance of 19,618/20,098 (97.612%) and zero API, validation, pager, or slot-erase failures.

The sampled GPQA, MMLU-Pro, GSM8K, and IFEval rows are not full-suite or leaderboard-comparable claims. ToolEval is a local functional rubric and recorded injection/safety misses in standard TC33/TC34 and hard TC81; do not use the aggregate as evidence for unsupervised tool safety.

Performance

Version 1.0/H121 matched performance row; version 1.1 changes cache-state correctness rather than kernels or model execution:

Prompt + generation	Cold prefill	Generation	TTFP	MTP accepted
8,192 + 128	359.4267 tok/s	30.7970 tok/s	22.807 s	84/128

Cold context ladder, same artifacts on the pre-fix H111 runtime, 128 generated tokens per row:

Prompt tokens	Prefill tok/s	Generation tok/s
512	298.3149	29.9648
2,048	376.9801	24.3850
4,096	378.9664	22.8458
8,192	372.8099	32.1142
16,384	295.0188	28.3220
32,768	228.2670	23.2864
65,536	174.6093	14.7909
131,072	121.0477	11.3400

All context rows were exact-count, real-source, cold prompts with no logged error. H121 changes MTP continuation-buffer lifetime, not prefill, but only the 8K release row has been repeated post-fix. The configured context is 262,144; the tested ladder stops at 131,072.

BF16 and Q5 diagnostic comparison

On a small 64-position, four-domain full-vocabulary panel, the CIRU model tied the size-matched Q5 control at 59/64 (92.19%) BF16 top-token agreement. On the shared 60-token observed-next-token slice:

BF16 PPL: 2.017988
CIRU IU4 PPL: 2.242104 (+11.106%)
Q5 PPL: 2.341209 (+16.017%)

The CIRU arm had lower observed-token PPL on this tiny slice, while Q5 had better aggregate full-distribution forward KL (0.170264 vs 0.223406 nats). This is diagnostic evidence, not a broad perplexity benchmark.

Architecture and storage

The 1,223 target tensors are stored as:

Type	Tensors
F32	388
Q5_K	328
Q8_0	290
Q4_1	144
Q5_1	48
BF16	25

The 144 Q4_1 routed-expert tensors occupy 75,497,472,000 bytes. The protected core occupies 3,900,335,968 bytes. The external PLE payload is exact FP8 E4M3 with one BF16 scale. GGUF names were mechanically rewritten for this public release without requantizing tensor arrays; full evaluated-to-public identity mapping is in PROVENANCE.md.

Hardware and operational requirements

Intended: Ryzen AI Max+ 395 / Radeon 8060S with 128 GiB unified memory.
Fast NVMe is mandatory for the external PLE pager.
Allow at least 160 GiB free storage for model files and verification/work space.
Linux is required for the measured P16/O_DIRECT PLE prefill path.
The model is text-only; no projector is shipped.
Bind to loopback unless you add authentication, TLS, request limits, and a reverse proxy.

Lineage, license, and credit

Text lineage: Qwen/Qwen3.8-Flash-Next@f5d08274bafd880402bd16f5e3e6c514136ec06c
PLE lineage: Qwen/Qwen3.8-Flash-Next-FP8@bcd9f01ddc9cff2316eb84281bebcd5b058bddce
Runtime base: ggml-org/llama.cpp@f5e85d43a048f3d5adefb4c5e29867d8077fba62

Model artifacts are distributed under the included Qwen Community License 1.0. The runtime repository retains the MIT license and third-party notices of its components.

Thanks to Qwen, ggml-org and the llama.cpp community, Ryan Monsurate for Qwen MTP integration work adapted by the release runtime, AMD's open-source ROCm ecosystem, and the contributors identified in NOTICE.md.

CIRU is an independent community research project. AMD and Qwen marks belong to their respective owners; their appearance does not imply sponsorship or endorsement.

Citation

@software{ciru_qwen38_flash_strix_iu4_2026,
  title        = {Qwen3.8-Flash-CIRU-STRIX-IU4},
  author       = {{CIRU}},
  year         = {2026},
  month        = {8},
  url          = {https://huggingface.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4},
  version      = {1.1}
}

Downloads last month: 7,567

GGUF

Model size

126B params

Architecture

qwen4exp

Hardware compatibility

8-bit

View +1 variant

Model tree for jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

(164)

this model