Qwen3.5-2B-oQ4

This model was quantized using oQ mixed-precision quantization.

Quantization details

• Model type: qwen3_5
• Bits: 4
• Group size: 64
• Format: MLX safetensors

Benchmark

Model	File size	MMLU	JMMLU	HELLASWAG	ARC_CHALLENGE	GSM8K
Qwen3.5-0.8B-oQ6	830 MB	42.7%	30.0%	45.7%	62.3%	49.7%
Qwen3.5-0.8B-oQ8	978 MB	42.0%	28.0%	45.7%	59.3%	46.3%
Qwen3.5-0.8B	1.75 GB	42.0%	30.0%	39.7%	63.7%	47.3%
Qwen3.5-2B-oQ3.5	1.62 GB	42.3%	41.3%	31.3%	67.0%	40.3%
Qwen3.5-2B-oQ4	1.77 GB	48.3%	41.3%	53.7%	74.7%	59.0%

Detail

Model	Benchmark	Accuracy	Correct	Total	Time(s)
Qwen3.5-0.8B-oQ6	MMLU	42.7%	128	300	172.2
Qwen3.5-0.8B-oQ6	JMMLU	30.0%	90	300	133.5
Qwen3.5-0.8B-oQ6	HELLASWAG	45.7%	137	300	125.4
Qwen3.5-0.8B-oQ6	ARC_CHALLENGE	62.3%	187	300	113.4
Qwen3.5-0.8B-oQ6	GSM8K	49.7%	149	300	404
Qwen3.5-0.8B-oQ8	MMLU	42.0%	126	300	167.3
Qwen3.5-0.8B-oQ8	JMMLU	28.0%	84	300	169.1
Qwen3.5-0.8B-oQ8	HELLASWAG	45.7%	137	300	129.7
Qwen3.5-0.8B-oQ8	ARC_CHALLENGE	59.3%	178	300	115
Qwen3.5-0.8B-oQ8	GSM8K	46.3%	139	300	363.9
Qwen3.5-0.8B	MMLU	42.0%	126	300	156.4
Qwen3.5-0.8B	JMMLU	30.0%	90	300	202.6
Qwen3.5-0.8B	HELLASWAG	39.7%	119	300	121.4
Qwen3.5-0.8B	ARC_CHALLENGE	63.7%	191	300	113.6
Qwen3.5-0.8B	GSM8K	47.3%	142	300	473.9
Qwen3.5-2B-oQ3.5	MMLU	42.3%	127	300	271.7
Qwen3.5-2B-oQ3.5	JMMLU	41.3%	124	300	140
Qwen3.5-2B-oQ3.5	HELLASWAG	31.3%	94	300	156
Qwen3.5-2B-oQ3.5	ARC_CHALLENGE	67.0%	201	300	129.4
Qwen3.5-2B-oQ3.5	GSM8K	40.3%	121	300	776.8
Qwen3.5-2B-oQ4	MMLU	48.3%	145	300	238.5
Qwen3.5-2B-oQ4	JMMLU	41.3%	124	300	122.4
Qwen3.5-2B-oQ4	HELLASWAG	53.7%	161	300	139.2
Qwen3.5-2B-oQ4	ARC_CHALLENGE	74.7%	224	300	115.4
Qwen3.5-2B-oQ4	GSM8K	59.0%	177	300	718.3