GBP-DE

/

Qwen3.8-Flash-Next-oQ5e-mtp

Like 10

Thanks for posting this benchmark! I ran the same oMLX context benchmark with the same Qwen3.8-Flash-Next-oQ5e-mtp model on my M5 Max with 128 GB unified memory, using 256k as the target and “Max Context” prefill priority.

I got 81,920 verified prefill tokens, limited by available memory, at 1,268 tok/s (2m51s).

Interestingly, in a real multi-turn coding session I managed to grow the context to around 193k tokens before eventually hitting a memory error, presumably thanks to cache reuse between turns. So the benchmark’s cold/full-prefill limit and the practically reachable size of a growing cached conversation seem to be very different things.

Unfortunately this also shows how tight oQ5e is on a 128 GB machine. I’d prefer not to drop to Q4 because the quality loss is noticeable to me. Looks like I need more unified memory. 🙂

我用opencode编程上下文真的只有5w，kv缓存和上下文压缩也没起作用哈哈，Q4量化模型基础知识损失很大相比于Q5很差了，早知道攒攒钱买m5ultra512G了😭

终于发现问题，我的opencode为什么只有5w上下文窗口，omlx基准测试后会直接修改配置文件限制模型上下文

Thanks, now I understand what happened. The 57,344 tokens weren’t your actual memory ceiling - the first oMLX benchmark had written that value into max_context_window, so OpenCode simply rejected anything above it. Your later 128k benchmark successfully verified the full 131,072 tokens and was limited by the selected target, so your actual ceiling is clearly higher.

I tested the same oQ5e model on my M5 Max with 128 GiB: 81,920 tokens verified, explicitly limited by available memory, at 1,268 tok/s. Interestingly, in a growing cached coding session I still managed to reach around 193k tokens before finally hitting a memory error.

And I completely agree about Q4 vs Q5. Q4 is much easier on memory, but the quality loss is too noticeable for me. At this exact moment I really wish I had the big M5 Ultra with 256 GiB. 😭 With that, oQ6e should be comfortable and oQ8e would probably be genuinely usable too. That feels like the point where you could stop fighting memory limits and actually do more serious and useful work with these models - and have a lot more fun doing it. 😅

By the way, what Mac are you using?

哈哈我的是M1 Ultra 128G老古董了，我发现Q5这模型就设置到131072tokens，opencode上下文5w就执行压缩是最舒服的，能力很强大

Ah, that’s interesting! I’ve made a note of your setup and I’ll keep an eye on it during my next coding experiments. 131k context with compression starting around 50k sounds like a very sensible sweet spot for Q5 on 128 GiB.

I really wish this model wasn’t around 180B parameters, but something closer to 150B. Then oQ5e would probably fit almost perfectly for both of us — and for a lot of other 128 GiB Mac users — with much more breathing room for context. 😅

So… where exactly do we send our wishlist to the Qwen team? 😂 Maybe we just need to ask very nicely for a 150B Flash-Next.

完全可以，我这就发邮件哈哈，我认为这个对阿里qwen实验室来说并不困难，他们的kl散度白盒蒸馏技术挺强的，以后大模型趋势也是越小越好我觉得

Haha, please do! 😄 If they actually make a ~150B Flash-Next because of your email, I’ll owe you a beer.

I agree that smaller and more efficient models are the right direction - but only up to a point. Fewer parameters can also mean less knowledge and capacity. A brilliant 27B model doesn’t help me much if I constantly have to compensate for knowledge it simply doesn’t contain. For local use, I still want a model with a really substantial amount of knowledge inside it.

Even something like an excellent ~80B A10 model would already be very interesting. But my dream would be around 150B, definitely not dense 😂, with Qwen’s architecture and distillation doing as much work as possible. If most of the capability and knowledge of the larger model could survive in ~150B, running that comfortably at Q5 or maybe even close to Q6 (with few content for some special tasks) on 128 GiB would be fantastic.

That’s really the sweet spot I’m hoping for: smaller and more efficient, but not smaller at the cost of becoming noticeably less knowledgeable or capable. 😄

I did some more testing with oMLX 0.7.0.dev4 today, and you might want to try this too. 😄

I still had the original 335 GB Qwen3.8-Flash-Next, so I quantized it myself with dev4’s new layer-streaming oQe calibration — first to oQ5e and then to oQ4e.

The surprising result is that the new oQ4e is much better than the older Q4 experience I complained about before. On my M5 Max 128 GiB, the built-in context benchmark now completes the full 262,144 tokens with the new oQ4e. My old HF oQ5e reaches about 104k, while my newly generated dev4 oQ5e only reaches about 72–74k.

I then gave all three models exactly the same large 45-part reasoning test with calculations, multi-year dependencies, traps between income/assets/realized gains, consistency checks and a long output.

dev4 oQ4e: 60.1 tok/s generation, 502.6 s total
dev4 oQ5e: 48.9 tok/s, 585.9 s total
old HF oQ5e: 40.9 tok/s, 664.2 s total

All three essentially solved the entire test correctly. The old oQ5e caught one subtle ambiguity slightly better, so I still think Q5 has a little more quality reserve — but the difference was surprisingly small. The new dev4 oQ4e was much faster, stayed completely coherent through the long answer, and gives me the full 256k context.

So if you still have the original model, maybe try generating a fresh oQ4e with dev4 + oQe calibration yourself. On a 128 GiB Mac this suddenly looks like a very interesting sweet spot. I didn’t expect Q4 to convince me either. 😂