rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp performance in oMLXL

by opnsrcntrbtrian - opened Aug 2

Discussion

opnsrcntrbtrian

Aug 2

•

edited Aug 2

oMLX - LLM inference, performance benchmark Macbook Pro M4 48 GB unified RAM

Benchmark Model: Qwen3.6-35B-A3B-oQ6e-mtp
Engine: Force mlx-lm

Single Request Results

Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem
pp1024/tg128 1230.5 22.62 832.2 tok/s 44.6 tok/s 4.113 280.1 tok/s 28.62 GB
pp4096/tg128 4857.4 76.81 843.3 tok/s 19.5 tok/s 5.069 808.7 tok/s 29.45 GB
pp8192/tg128 10053.7 19.15 814.8 tok/s 52.6 tok/s 12.500 665.6 tok/s 29.80 GB
pp16384/tg128 21613.0 20.06 758.1 tok/s 50.2 tok/s 24.175 683.0 tok/s 30.45 GB
pp32768/tg128 52608.2 23.89 622.9 tok/s 42.2 tok/s 55.657 591.0 tok/s 31.79 GB

opnsrcntrbtrian changed discussion status to closed Aug 2

opnsrcntrbtrian changed discussion title from rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp error in oMLXL to rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp in oMLXL Aug 2

opnsrcntrbtrian changed discussion title from rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp in oMLXL to rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp performance in oMLXL Aug 2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

rosingrind/Qwen3.6-35B-A3B-oQ6e-mtp performance in oMLXL

Benchmark Model: Qwen3.6-35B-A3B-oQ6e-mtpEngine: Force mlx-lm

Single Request Results

Benchmark Model: Qwen3.6-35B-A3B-oQ6e-mtp
Engine: Force mlx-lm