---
base_model: Qwen/Qwen3.8-2.4T-A95B
license: other
license_name: qwen3.8-max
license_link: https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/main/LICENSE
language:
- en
- ja
tags:
- gguf
- qwen3.8
- moe
- reap
---

# Qwen3.8 REAP30 Width50 UD GGUF

[日本語](./README-ja.md)

An experimental reduced GGUF build of [Qwen3.8-2.4T-A95B](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B).

The routed-expert count is reduced by about 30%, then the intermediate width of each retained expert is reduced by 50%. The number of experts selected per token remains 10.

## Calibration and selection

The calibration set contains 97 completed English and Japanese samples covering general chat and reasoning, code generation, and tool use. It contains 94,874 profiled tokens, of which 78,061 are semantic assistant tokens used for scoring. Prompt and structure tokens are excluded.

### REAP30: expert axis

For expert `e`, the score is:

```text
REAP(e) = mean[t routed to e](
  top-10-renormalized router weight(t,e)
  × ||unweighted expert output(t,e)||₂
)
```

Experts are ranked independently in each of the 92 MoE layers. The top 358 of 512 are retained. No experts are reserved manually.

### Width50: intermediate axis

After the REAP30 slice, the reduced model is profiled again with the same calibration set. Every intermediate channel is scored independently for each expert:

```text
a(t,e,c)     = SiLU(gate(t,e,c)) × up(t,e,c)
score(e,c)   = Σ[t routed to e](router_weight(t,e) × a(t,e,c))²
               × ||Wdown,e[:,c]||₂²
```

The top 1,024 of 2,048 channels are retained for each expert. Low-coverage experts use a blend of the activation score and a static weight-energy prior.

## GGUF changes

| | Original | This build |
|---|---:|---:|
| MoE layers | 92 | 92 |
| Routed experts per layer | 512 | 358 |
| Routed expert FFN width | 2,048 | 1,024 |
| Experts selected per token | 10 | 10 |

The router and grouped expert tensors are sliced with the same per-layer expert map. Shared experts are retained. MTP tensors are omitted.

Width selection is channel-level rather than 256-channel-block-level. Retained gate/up rows are copied directly. `ffn_down_exps.weight` is dequantized, gathered by the selected channel map, packed into new QK256 blocks, and requantized to the source GGUF type.

Both folders use the same REAP and Width rankings.

## Files

| Folder | Shards | Size |
|---|---:|---:|
| `UD-IQ1_S` | 12 | about 184.72 GiB |
| `UD-Q1_0` | 10 | about 148.54 GiB |

Load the first shard in the selected folder. Each folder also includes `slice_manifest.json` with the applied ranking hashes and slice settings.

## Usage

`UD-Q1_0` uses the experimental IQ1_XXXS GGML type. Use the [iq1-narrow branch](https://github.com/unslothai/llama.cpp/tree/iq1-narrow):

```bash
git clone --branch iq1-narrow https://github.com/unslothai/llama.cpp
cmake -S llama.cpp -B llama.cpp/build \
  -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build llama.cpp/build -j --target llama-server
```

Example for a 32 GB GPU while keeping MoE weights on CPU:

```bash
./llama.cpp/build/bin/llama-server \
  -m ./UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.gguf \
  --cpu-moe \
  -ngl 80 \
  --ctx-size 8192 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0
```

This configuration was tested with 512 GB system RAM and one 32 GB RTX 5090. It offloaded 80 of 92 layers and used about 30.6 GiB of VRAM. Adjust `-ngl` and context size for your hardware.

This is a heavily reduced experimental model. Quality and stability may differ from the original model and depend on how closely the calibration distribution matches the target workload.