---
license: apache-2.0
base_model: stepfun-ai/Step-3.7-Flash
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
- gguf
- rocmfp4
- strix-halo
- gfx1151
- amd
- rocm
- llama.cpp
- moe
- vision
---

# Step-3.7-Flash — Q4_0_ROCMFP4_STRIX_LEAN GGUF

This is a ROCmFP4 "Strix Lean" quant of [stepfun-ai/Step-3.7-Flash](https://huggingface.co/stepfun-ai/Step-3.7-Flash), built and tested on a Ryzen AI Max+ 395 (Strix Halo, gfx1151, 128 GB unified memory, ROCm 7.2.4).

I run a lot of big models on Strix Halo hardware, and the ROCmFP4 format has been the best decode-speed lever I've found for it. Every ROCmFP4 build I could find was Qwen, Ornith, Laguna or Tess class, so I made one for Step-3.7-Flash. As far as I can tell this is the first one.

**Short version: it's faster than IQ4_XS at both ends — about 18–20% more decode throughput and 8–12% more prefill — with equivalent output quality. It costs about 10% more disk.**

## Requires the ROCmFP4 fork

`Q4_0_ROCMFP4_STRIX_LEAN` is not a stock llama.cpp quant type. **These files will not load on upstream llama.cpp, Ollama, LM Studio, or anything else that hasn't got the ROCmFP4 kernels.** You need a build of the fork:

```bash
git clone https://github.com/charlie12345/ROCmFPX
cd ROCmFPX
HIPCXX=$(hipconfig -l)/clang HIP_PATH=$(hipconfig -R) cmake -B build \
  -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_HIP_ROCWMMA_FATTN=ON \
  -DGGML_HIP_NO_VMM=ON -DGGML_HIP_MMQ_MFMA=ON -DCMAKE_BUILD_TYPE=Release \
  -DGGML_VULKAN=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j 4
```

Keep the job count low. On a 128 GB unified-memory box the HIP flash-attention kernels spike RAM hard at `-j 8` or higher and will OOM the machine.

## Files

| File | Size |
|---|---|
| `Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00001-of-00003.gguf` | 41.7 GiB |
| `Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00002-of-00003.gguf` | 41.5 GiB |
| `Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00003-of-00003.gguf` | 14.6 GiB |
| `mmproj-Step-3.7-Flash-f16.gguf` | 3.7 GiB |

97.7 GiB total for the model, 4.26 BPW. Point llama.cpp at the **first** shard and it picks up the rest automatically. The `mmproj` file is the original f16 vision projector from StepFun — you only need it if you want image input.

## Running it

```bash
env LD_LIBRARY_PATH=/path/to/ROCmFPX/build/bin:/opt/rocm/lib \
    HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
  /path/to/ROCmFPX/build/bin/llama-server \
  --host 127.0.0.1 --port 8080 \
  --model Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00001-of-00003.gguf \
  --mmproj mmproj-Step-3.7-Flash-f16.gguf \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --ctx-size 32768 --cache-type-k q8_0 --cache-type-v q8_0 \
  --parallel 1 --temp 0.6 --top-p 0.95 --reasoning-budget 512
```

Three things in there are not optional, and each one cost me time before I worked it out:

**`-dio`.** Without direct I/O the mmap path crawls up to around 120 GB RSS and sits there looking hung. It isn't hung and the file isn't corrupt, it's just the loader. With `-dio` it cold-loads in about 60 seconds.

**The env block.** `HSA_OVERRIDE_GFX_VERSION=11.5.1` and unified memory are needed for gfx1151, and putting the fork's `build/bin` first on `LD_LIBRARY_PATH` avoids a soname clash if you also have a Vulkan llama.cpp build on the box.

**`--reasoning-budget 512`.** Step-3.7-Flash thinks a lot. Left alone it will spend your entire `max_tokens` on chain-of-thought and hand you back an empty `content` field with `finish_reason: length`, which looks exactly like a broken quant. It isn't. Cap the thinking and the answer lands. 512 works well for chat; raise it for hard reasoning work.

## Benchmarks

Both quants cold-loaded one at a time on the same machine, same binary, same environment, same sampling. Every run generated **exactly 256 tokens** so decode numbers are actually comparable, and every prompt started with a unique nonce so nothing was served out of the prefix cache. Three runs each, medians below.

| | **This quant** | UD-IQ4_XS | Difference |
|---|---:|---:|---:|
| Prefill @ 8K | 331 tok/s | 296 tok/s | **+12%** |
| Decode @ 8K | 18.8 tok/s | 15.9 tok/s | **+18%** |
| Prefill @ 32K | 283 tok/s | 261 tok/s | **+8%** |
| Decode @ 32K | 14.9 tok/s | 12.4 tok/s | **+20%** |
| Size | 97.7 GiB | 88.8 GiB | +10% |

Compared against [unsloth/Step-3.7-Flash-GGUF](https://huggingface.co/unsloth/Step-3.7-Flash-GGUF) `UD-IQ4_XS`, tested at 32K context with q8_0 KV cache.

**Be fair about the size column.** This is a ~4.26 BPW quant and UD-IQ4_XS is roughly 3.9 BPW, so it's a higher-bit quant winning on speed while costing more disk. If you're comparing against Q4_K_M instead, ROCmFP4 normally comes out around 12–18% *smaller*. Against a sub-4-bit IQ quant it does not, and I'd rather say that up front than let the table imply a free lunch.

Quality spot checks came out equivalent, not identical — same correct answers on a palindrome-function task and a reasoning riddle, with different wording. Worth noting that IQ4_XS was noticeably more verbose in its chain-of-thought and needed a higher token budget before its final answer appeared, which is a practical point in this quant's favour if you're serving it.

## How it was made

```bash
llama-quantize --allow-requantize \
  Step-3.7-Flash-Q8_0-00001-of-000NN.gguf \
  Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN 8
```

**Quantized from the official Q8_0 GGUF, not from BF16.** The BF16 weights are 394 GB and there was no way to hold them alongside the output on this machine, so this is a requantization and `--allow-requantize` was required. I'd rather disclose that than have someone assume an F16 source. It measured and benched clean, but if you want a from-BF16 build, this isn't it.

Verified after quantizing: coherent output on a load proof before anything got deleted, the full benchmark run above, and image input confirmed working through the mmproj.

## Model

Step-3.7-Flash is StepFun's 198B MoE — 196B language model plus a 1.8B vision encoder, roughly 11B active parameters, 256K native context. Everything about the model's behaviour, prompt format and capabilities comes from upstream; see the [base model card](https://huggingface.co/stepfun-ai/Step-3.7-Flash). All I've changed is the numeric format.

Apache 2.0, same as the base model.

## Credit

The ROCmFP4 format and kernels are not my work — that's the [ROCmFPX](https://github.com/charlie12345/ROCmFPX) fork. I built the quant, ran the benchmarks and wrote them down.