File size: 3,792 Bytes
a20cf70 3da77a2 bdd5527 3da77a2 a20cf70 3da77a2 bdd5527 a20cf70 3da77a2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
library_name: mlx
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-2B/blob/main/LICENSE
pipeline_tag: text-generation
tags:
- mlx
- oq
- quantized
base_model:
- Qwen/Qwen3.5-2B
---
# Qwen3.5-2B-oQ4
This model was quantized using [oQ](https://github.com/jundot/omlx) mixed-precision quantization.
## Quantization details
- **Model type**: qwen3_5
- **Bits**: 4
- **Group size**: 64
- **Format**: MLX safetensors
# Benchmark
| Model | File size | MMLU | JMMLU | HELLASWAG | ARC_CHALLENGE | GSM8K |
| ------------------------------------ | --------- | ----- | ----- | --------- | ------------- | ----- |
| Qwen3.5-0.8B-oQ6 | 830 MB | 42.7% | 30.0% | 45.7% | 62.3% | 49.7% |
| Qwen3.5-0.8B-oQ8 | 978 MB | 42.0% | 28.0% | 45.7% | 59.3% | 46.3% |
| Qwen3.5-0.8B | 1.75 GB | 42.0% | 30.0% | 39.7% | 63.7% | 47.3% |
| Qwen3.5-2B-oQ3.5 | 1.62 GB | 42.3% | 41.3% | 31.3% | 67.0% | 40.3% |
| Qwen3.5-2B-oQ4 | 1.77 GB | 48.3% | 41.3% | 53.7% | 74.7% | 59.0% |
## Detail
| Model | Benchmark | Accuracy | Correct | Total | Time(s) |
| ------------------------------------ | ------------- | -------- | ------- | ----- | ------- |
| Qwen3.5-0.8B-oQ6 | MMLU | 42.7% | 128 | 300 | 172.2 |
| Qwen3.5-0.8B-oQ6 | JMMLU | 30.0% | 90 | 300 | 133.5 |
| Qwen3.5-0.8B-oQ6 | HELLASWAG | 45.7% | 137 | 300 | 125.4 |
| Qwen3.5-0.8B-oQ6 | ARC_CHALLENGE | 62.3% | 187 | 300 | 113.4 |
| Qwen3.5-0.8B-oQ6 | GSM8K | 49.7% | 149 | 300 | 404 |
| Qwen3.5-0.8B-oQ8 | MMLU | 42.0% | 126 | 300 | 167.3 |
| Qwen3.5-0.8B-oQ8 | JMMLU | 28.0% | 84 | 300 | 169.1 |
| Qwen3.5-0.8B-oQ8 | HELLASWAG | 45.7% | 137 | 300 | 129.7 |
| Qwen3.5-0.8B-oQ8 | ARC_CHALLENGE | 59.3% | 178 | 300 | 115 |
| Qwen3.5-0.8B-oQ8 | GSM8K | 46.3% | 139 | 300 | 363.9 |
| Qwen3.5-0.8B | MMLU | 42.0% | 126 | 300 | 156.4 |
| Qwen3.5-0.8B | JMMLU | 30.0% | 90 | 300 | 202.6 |
| Qwen3.5-0.8B | HELLASWAG | 39.7% | 119 | 300 | 121.4 |
| Qwen3.5-0.8B | ARC_CHALLENGE | 63.7% | 191 | 300 | 113.6 |
| Qwen3.5-0.8B | GSM8K | 47.3% | 142 | 300 | 473.9 |
| Qwen3.5-2B-oQ3.5 | MMLU | 42.3% | 127 | 300 | 271.7 |
| Qwen3.5-2B-oQ3.5 | JMMLU | 41.3% | 124 | 300 | 140 |
| Qwen3.5-2B-oQ3.5 | HELLASWAG | 31.3% | 94 | 300 | 156 |
| Qwen3.5-2B-oQ3.5 | ARC_CHALLENGE | 67.0% | 201 | 300 | 129.4 |
| Qwen3.5-2B-oQ3.5 | GSM8K | 40.3% | 121 | 300 | 776.8 |
| Qwen3.5-2B-oQ4 | MMLU | 48.3% | 145 | 300 | 238.5 |
| Qwen3.5-2B-oQ4 | JMMLU | 41.3% | 124 | 300 | 122.4 |
| Qwen3.5-2B-oQ4 | HELLASWAG | 53.7% | 161 | 300 | 139.2 |
| Qwen3.5-2B-oQ4 | ARC_CHALLENGE | 74.7% | 224 | 300 | 115.4 |
| Qwen3.5-2B-oQ4 | GSM8K | 59.0% | 177 | 300 | 718.3 | |