---
library_name: mlx
base_model: tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2
tags:
  - mlx
  - japanese
  - qwen3
  - swallow
  - apple-silicon
  - moe
  - 4bit
  - quantized
license: apache-2.0
language:
  - ja
  - en
pipeline_tag: text-generation
---

# Qwen3-Swallow-30B-A3B-SFT-v0.2 — MLX 4bit

MLX 4bit quantized version of [tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2](https://huggingface.co/tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2) for Apple Silicon Macs.

## Model Details

- **Original model**: [tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2](https://huggingface.co/tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2)
- **Architecture**: Mixture of Experts (MoE) — 30B total parameters, 3B active parameters
- **Training**: Japanese Continued Pre-Training + Supervised Fine-Tuning by [Tokyo Institute of Technology Swallow Project](https://swallow-llm.github.io/qwen3-swallow.en.html)
- **License**: Apache 2.0

## Conversion Details

| Item | Value |
|------|-------|
| Conversion tool | [mlx-lm](https://github.com/ml-explore/mlx-lm) |
| Quantization | 4bit |
| Model size | ~17 GB |
| Source | `tokyotech-llm/Qwen3-Swallow-30B-A3B-SFT-v0.2` |

## Performance (MacBook Pro M4 Max, 128GB)

| Metric | Value |
|--------|-------|
| Generation speed | 120.6 tokens/s |
| Peak memory usage | 17.3 GB |

## Usage

### Install

```bash
pip install mlx-lm
```

### Text Generation

```bash
mlx_lm.generate \
  --model tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit \
  --prompt "生成AIについて、10歳向けの説明をして" \
  --max-tokens 500
```

### Chat

```bash
mlx_lm.chat --model tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit
```

### OpenAI-Compatible API Server

```bash
mlx_lm.server \
  --model tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit \
  --port 8080
```

### Python API

```python
from mlx_lm import load, generate

model, tokenizer = load("tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit")
messages = [{"role": "user", "content": "日本の四季の魅力を説明して"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=500))
```

## Recommended Hardware

| Machine | Memory | Status |
|---------|--------|--------|
| M4 Max 128GB | Plenty of headroom | ✅ |
| M4 Pro 64GB | Comfortable | ✅ |
| M4 Pro 48GB | Comfortable | ✅ |
| M1/M2/M3 16GB | Tight | ⚠️ |

## Other Variants

| Precision | Repository | Size | Speed |
|-----------|-----------|------|-------|
| **4bit (this model)** | [tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit](https://huggingface.co/tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-4bit) | 17 GB | 120.6 tok/s |
| 8bit | [tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-8bit](https://huggingface.co/tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-8bit) | 32 GB | 89.7 tok/s |
| fp16 | [tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-fp16](https://huggingface.co/tocchitocchi/Qwen3-Swallow-30B-A3B-SFT-v0.2-MLX-fp16) | 61 GB | 60.9 tok/s |

## Compatible Tools

- [mlx-lm](https://github.com/ml-explore/mlx-lm) (CLI / Python / API server)
- [LM Studio](https://lmstudio.ai/) (GUI app)
- [Pico AI Server](https://apps.apple.com/app/pico-ai-server/id6502491449) (App Store)