Qwopus3.5-9B-Coder-oQ4-mtp

This model was quantized using oQ (oMLX v0.5.1) mixed-precision quantization.

Changelog

• 2026-07-19: Chat template upgraded to froggeric/Qwen-Fixed-Chat-Templates

Quantization details

• Model type: qwen3_5
• Bits: 4
• Group size: 64
• Format: MLX safetensors

Environment

• Hardware: M5 MacBook Air 32GB
• Inference Framework: oMLX v0.5.1
• Max Concurrent Requests: 4
• Settings:
  • Thinking: Disabled
  • Chat template parameter: enable_thinking=false (forced)
  • TurboQuant KV Cache: Enabled (disable before oMLX v0.5.2.dev1)
  • Lightning MTP: Enabled (key speed improvement)

Performance Benchmarks

Note: Results are for reference only and may vary depending on hardware, software configuration, and workload.

Single Request Results

Test	TTFT(ms)	TPOT(ms)	pp TPS	tg TPS	E2E(s)	Throughput	Peak Mem
pp1024/tg128	1735.2	26.43	590.1 tok/s	38.1 tok/s	5.092	226.2 tok/s	5.96 GB
pp4096/tg128	6142.9	28.49	666.8 tok/s	35.4 tok/s	9.761	432.7 tok/s	6.57 GB

Continuous Batching (pp1024 / tg128)

Batch	tg TPS	Speedup	pp TPS	pp TPS/req	TTFT(ms)	E2E(s)
1x	38.1 tok/s	1.00x	590.1 tok/s	590.1 tok/s	1735.2	5.092
2x	48.4 tok/s	1.27x	606.6 tok/s	303.3 tok/s	3376.0	8.662
4x	53.2 tok/s	1.40x	578.0 tok/s	144.5 tok/s	6889.9	16.712

Intelligence Benchmark

Note: Each benchmark round tests only 30 questions. Results are for reference only.

Benchmark	Accuracy	Correct	Total	Time(s)	Think
MMLU	80.0%	24	30	57.1	No
TRUTHFULQA	86.7%	26	30	24.4	No
GSM8K	83.3%	25	30	355.2	No
MATHQA	40.0%	12	30	25.6	No
HUMANEVAL	80.0%	24	30	157.7	No