vllm serving issue
Hello, first of all thank you for such great models. Im facing issue while serving on vllm and constantly getting error in disabling thinking:
(APIServer pid=1) WARNING 02-25 15:46:28 [protocol.py:51] The following fields were present in the request but ignored: {'enable_thinking', 'extra_body'}
I'm using this docker image: docker run -d --gpus all
--name vllm-qwen35-27b
--ipc=host
--shm-size 16g
-v ~/hf_cache:/root/.cache/huggingface
-p 8001:8000
vllm/vllm-openai:qwen3_5
Qwen/Qwen3.5-27B
--host 0.0.0.0
--port 8000
--tensor-parallel-size 1
--max-model-len 20000
--gpu-memory-utilization 0.85
--enable-prefix-caching
--disable-log-stats
--reasoning-parser qwen3
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
How can I disable reasoning?
If you are using the latest vllm for inference, according to vllm document: https://docs.vllm.ai/en/latest/features/reasoning_outputs/#disabling-thinking-mode-by-default
You can disable reasoning on the server-side by
vllm serve Qwen/Qwen3-8B \
--reasoning-parser qwen3 \
--default-chat-template-kwargs '{"enable_thinking": false}'
If you are using the latest vllm for inference, according to vllm document: https://docs.vllm.ai/en/latest/features/reasoning_outputs/#disabling-thinking-mode-by-default
You can disable reasoning on the server-side by
vllm serve Qwen/Qwen3-8B \ --reasoning-parser qwen3 \ --default-chat-template-kwargs '{"enable_thinking": false}'
thanks, but I found out that vllm inference is not optimal and that sglang manages this much better
you just disable reasoning by sending with extra body
response = client.chat.completions.create(
model="Qwen/Qwen3.5-35B-A3B",
messages=messages,
max_tokens=81920,
temperature=1.0,
top_p=0.95,
presence_penalty=1.5,
extra_body={
"top_k": 20,
"mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
},
)
which works for all and in your parameter name is wrong --name vllm-qwen35-27b, it should be 3.5-27b