---
license: other
license_name: qwen
license_link: https://huggingface.co/Qwen/Qwen3.5-122B-A10B-FP8
base_model:
  - Intel/Qwen3.5-122B-A10B-int4-AutoRound
  - Qwen/Qwen3.5-122B-A10B-FP8
pipeline_tag: text-generation
tags:
  - qwen3_5_moe
  - dgx-spark
  - gb10
  - vllm
  - speculative-decoding
  - dflash
  - int4
  - fp8
---

# Qwen3.5-122B-A10B — hybrid INT4 + FP8 (DGX Spark)

A hybrid-quantized checkpoint of **Qwen3.5-122B-A10B** for bandwidth-optimal
decode on the **NVIDIA DGX Spark (GB10 / SM121, 128 GB unified)** under vLLM.
Routed experts and attention stay INT4; the BF16 **shared experts** (dense — read
on every token) are replaced with **calibrated FP8** from the official FP8 release.

## Composition
- **Routed experts + attention:** INT4 (GPTQ / AutoRound) — from [`Intel/Qwen3.5-122B-A10B-int4-AutoRound`](https://huggingface.co/Intel/Qwen3.5-122B-A10B-int4-AutoRound).
- **Shared experts:** FP8 E4M3 block-128 — from [`Qwen/Qwen3.5-122B-A10B-FP8`](https://huggingface.co/Qwen/Qwen3.5-122B-A10B-FP8).
- Embeddings / norms / head: unchanged from the INT4 base.

144 shared-expert layers convert BF16→FP8 — an always-on bandwidth lever worth
**+28 % at base / no-spec decode** on GB10 (28.2 → 36.0 tok/s), diminishing under
high-acceptance speculative decode (see the project's amortization law).

## Usage
Built for and served by
**[qwen3.5-122B-A10B-on-spark](https://github.com/Entrpi/qwen3.5-122B-A10B-on-spark)**
(`--profile dense`). It requires that project's vLLM 0.23 hybrid-FP8 dispatch
patch — an `INCConfig.maybe_update_config` override that dispatches
`Fp8LinearMethod` for the FP8 shared-expert layers. Stock vLLM will not dispatch
the mixed INT4/FP8 scheme.

## Provenance & credit
Recipe (hybrid INT4+FP8, INT8 lm-head, MTP) and build script:
[`albond/DGX_Spark_Qwen3.5-122B-A10B-AR-INT4`](https://github.com/albond/DGX_Spark_Qwen3.5-122B-A10B-AR-INT4).
Built with `build-hybrid-checkpoint.py`, which merges FP8 non-expert tensors into
the INT4 checkpoint.

## License
Derived work — inherits the upstream **Qwen3.5-122B-A10B** and **Intel AutoRound**
terms. See the base-model repositories linked above.