Why the reasoning so much?
Is there any issue with the model?
My flag is:
.\llama-server.exe -m "D:\AI Models\Local_Models\Qwen\Qwen3.8-27B\Qwen3.8-27B-UD-Q5_K_XL.gguf"
-c 65536 -np 1
-ngl 99 -ts 18,10
-sm layer -mg 0
-fa on -b 2048
-ub 1024 --cache-type-k q4_0
--cache-type-v q4_0 --jinja
--reasoning-preserve --temp 0.6
--top-p 0.95 --top-k 20
--min-p 0.0 --presence-penalty 0.0
--repeat-penalty 1.08 --port 8080
--spec-default --spec-type draft-mtp
--agent
Qwen3.8 comes with official support for reasoning_effort, which can be used to adjust reasoning depth and control cost:
xhigh (default): for complex tasks demanding thorough analysis
medium: balancing accuracy and speed
low: efficient reasoning optimizing for speed and cost
Qwen3.8 comes with official support for reasoning_effort, which can be used to adjust reasoning depth and control cost:
xhigh (default): for complex tasks demanding thorough analysis
medium: balancing accuracy and speed
low: efficient reasoning optimizing for speed and cost
Thanks for letting me know!
Any one knows how to force reasoning to medium in llama.cpp?
for low:
--chat-template-kwargs '{"preserve-thinking": true, "reasoning_effort": "low"}'
for medium:
--chat-template-kwargs '{"preserve-thinking": true, "reasoning_effort": "medium"}'
Mhh, choosing xhighas a default feels strange. Benchmaxxing?
Even on low the reasoning trace is bananas. It just spent 5k tokens thinking about how to implement a simple Flappy Bird game in HTML5.
The reasoning effort is absolutely crazy yes...Normal chat is like 1200-1500 tokens for a simple conversation (Hermes Agent). Even with reasoning disabled (/reasoning none) I have the feeling it generates way more tokens per answer than 3.6....
medium seems like the sweet spot for me so far.
It generates more reasoning tokens, but then it manages to complete things in fewer turns.
When it does manage to complete the results are indeed really good. DeepSeek V4 Flash levels of quality. However it doesn't always manage to get to the end. I'm having issues with agentic coding where it confuses itself. It will write a perfectly fine file then think about it and say "oh, it looks like I made a mistake, I wrote X. Let me rewrite it" where X is a mistake that does not even exist in the file. And it didn't happen once. I had to stop it and tell it the file is ok, just move on.
I guess I just need to figure out the best way to work with it 😃
When it does manage to complete the results are indeed really good. DeepSeek V4 Flash levels of quality. However it doesn't always manage to get to the end. I'm having issues with agentic coding where it confuses itself. It will write a perfectly fine file then think about it and say "oh, it looks like I made a mistake, I wrote X. Let me rewrite it" where X is a mistake that does not even exist in the file. And it didn't happen once. I had to stop it and tell it the file is ok, just move on.
I guess I just need to figure out the best way to work with it 😃
try this: --temp 0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0