---
base_model: ukisai/Swift-Qwen3.8-27b
base_model_relation: quantized
license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b#license-and-access
pipeline_tag: image-text-to-text
tags:
  - mlx
  - mtplx
  - qwen3_8
  - qwen3_5
  - apple-silicon
  - 4-bit
  - mtp
  - speculative-decoding
---

# Swift-Qwen3.8-27B — MTPLX 4-bit

MTPLX / MLX conversion of `ukisai/Swift-Qwen3.8-27b`, built for fast local inference on Apple Silicon.

This build was created with **MTPLX Forge** and keeps Swift's native **MTP head in BF16** for self-speculative decoding.

The model body is quantized to **4-bit affine, group size 64**.

## Highlights

- Base model: `ukisai/Swift-Qwen3.8-27b`
- Runtime: MTPLX / MLX
- Body quantization: 4-bit affine
- Group size: 64
- MTP head: BF16
- Maximum MTP depth: D3
- Vision weights: included
- Tested hardware: Apple M5 Max
- Tested memory: 128 GB unified memory
- MTPLX version used for Forge verification: 2.11.2
- Published artifact size: ~16.9 GB

## Performance

Forge verification was run locally on an **Apple M5 Max with 128 GB unified memory**.

| Mode | Mean MTP acceptance | Decode speed | Speedup vs AR |
|---|---:|---:|---:|
| AR baseline | — | 29.4 tok/s | 1.00× |
| MTP D1 | 96% | 46.5 tok/s | 1.58× |
| MTP D2 | 98% | 61.9 tok/s | 2.11× |
| **MTP D3** | **93%** | **65.2 tok/s** | **2.22×** |

**Recommended depth on this machine: D3.**

The D3 result improves measured decode throughput from **29.4 tok/s to 65.2 tok/s**, a **2.22× speedup** over autoregressive decoding.

### Verification sampler

```text
temperature = 1.0
top_p      = 0.95
top_k      = 20
```

The complete Forge verification data is included in:

```text
mtplx_runtime.json
```

### Benchmark scope

These numbers measure **decode throughput for this MTPLX conversion**.

They should not be interpreted as:

- an end-to-end Swift-vs-Qwen3.8 task-speed benchmark;
- a quality comparison against the original BF16 checkpoint;
- a claim that every prompt will decode at 65.2 tok/s.

Actual throughput depends on prompt length, context size, generation length, sampling parameters, thermal conditions, and workload.

The Forge verification reported successful quality checks for AR, D1, D2 and D3.

## Quantization recipe

This checkpoint was built with:

```text
Body bits:       4-bit
Group size:      64
Quantization:    affine
Body dtype:      auto
MTP strategy:    keep BF16
MTP depth max:   3
```

In compact form:

```text
Swift-Qwen3.8-27B
       │
       ├── model body → MLX 4-bit affine / g64
       │
       ├── MTP head   → BF16
       │
       └── vision     → preserved
       │
       ▼
Swift-Qwen3.8-27B-MTPLX
```

## MTP

Swift retains the native multi-token prediction head from its Qwen3.8 architecture.

MTPLX can use this head as a self-speculative drafter: candidate tokens are predicted ahead and verified by the target model.

On the tested M5 Max, Forge selected **MTP depth 3** as the fastest configuration.

The MTP head is intentionally kept in **BF16** in this build rather than quantized.

## Vision

The Forge conversion contains the vision weights in:

```text
model-vision.safetensors
```

The published artifact therefore preserves the multimodal components of the source checkpoint.

The performance numbers above are text-generation decode measurements. Multimodal quality and image-processing throughput have not been independently benchmarked for this release.

## Installation

Install MTPLX on Apple Silicon:

```bash
brew install youssofal/mtplx/mtplx
```

Download this model:

```bash
mtplx pull arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX
```

Then start MTPLX:

```bash
mtplx start
```

Select this model when prompted.

You can also inspect the checkpoint before running it:

```bash
mtplx inspect arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX
```

## Retune for your Mac

The optimal MTP depth can depend on the Apple Silicon generation, memory bandwidth, thermal state and workload.

To benchmark AR vs D1/D2/D3 on your own machine:

```bash
mtplx tune \
  --model arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX \
  --retune
```

Do not assume D3 will necessarily be optimal on every Mac.

## API usage

MTPLX can expose the model through an OpenAI-compatible local API.

Start the server:

```bash
mtplx serve --port 8000
```

The default local endpoint is:

```text
127.0.0.1:8000
```

This can be used with clients supporting OpenAI-compatible chat-completion APIs.

## Files

Important files in this repository include:

```text
model-00001-of-00003.safetensors
model-00002-of-00003.safetensors
model-00003-of-00003.safetensors
model-vision.safetensors
mtp.safetensors
model.safetensors.index.json
mtplx_runtime.json
config.json
generation_config.json
chat_template.jinja
tokenizer.json
```

`mtp.safetensors` contains the BF16 MTP sidecar used for speculative decoding.

`mtplx_runtime.json` contains Forge provenance and the local verification results.

## Provenance

Source model:

```text
ukisai/Swift-Qwen3.8-27b
```

Source revision used by Forge:

```text
1b30aaaf753fe5c1cb51ada2ea0367a53445359c
```

Conversion:

```text
MTPLX Forge 2.11.2
```

This repository is a **quantized runtime conversion**. It is not a new fine-tune and does not alter the training performed by UkisAI.

## About Swift

Swift-Qwen3.8-27B is an efficient-thinking derivative of Qwen3.8-27B developed by UkisAI.

The purpose of this repository is only to provide an Apple-Silicon-oriented MTPLX/MLX conversion of that model.

For model architecture, training methodology, evaluation results, intended use and limitations, refer to the original model card:

```text
ukisai/Swift-Qwen3.8-27b
```

## License

This conversion inherits the licensing conditions of the original Swift checkpoint.

Swift weights are distributed under the **Swift Open License v1.0**.

Please review the **License and access** section of the original:

```text
ukisai/Swift-Qwen3.8-27b
```

before downloading, redistributing or using these weights commercially.

The MTPLX software itself is a separate project and has its own software license.

## Validation status

Validated:

- model loads through the MTPLX Forge verification pipeline;
- autoregressive decoding works;
- MTP D1 works;
- MTP D2 works;
- MTP D3 works;
- speculative acceptance remains high;
- D3 outperforms the AR baseline on the tested M5 Max;
- vision weights are present in the converted checkpoint.

Not yet independently evaluated for this conversion:

- full benchmark-suite parity with upstream BF16 Swift;
- long-context quality;
- multimodal benchmark quality;
- quantization-induced quality loss;
- end-to-end Swift-vs-Qwen3.8 task completion time.

Contributions and independent benchmark results are welcome.

## Credits

Original model:

```text
UkisAI — Swift-Qwen3.8-27B
```

Apple Silicon runtime and Forge tooling:

```text
MTPLX — Youssofal
```

MTPLX conversion and benchmark:

```text
arsis-dev
```

Thanks to the original model authors and the MTPLX project for making this conversion possible.