---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/LICENSE
pipeline_tag: text-generation
base_model: Qwen/Qwen3.5-122B-A10B
tags:
- mlx
---

[Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) optimized for MLX.
This quant does not support image input.

For vision support: https://huggingface.co/spicyneuron/Qwen3.5-122B-A10B-MLX-4.7bit-vision

**EDIT:** Updated chat template to enable better prompt caching.

# Usage

```sh
# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/Qwen3.5-122B-A10B-MLX-4.6bit
```

# Methodology

Quantized with a [mlx-lm fork](https://github.com/ml-explore/mlx-lm/pull/922), drawing inspiration from Unsloth/AesSedai/ubergarm style mixed-precision GGUFs.
MLX quantization options differ than llama.cpp, but the principles are the same:

- Sensitive layers like MoE routing, attention, and output embeddings get higher precision
- More tolerant layers like MoE experts get lower precision