lucianommartins commited on
Commit
5330c5f
·
verified ·
1 Parent(s): 14d9d4b

fix: restore model turn + thinking cue after tool responses

Browse files

The generation prompt suppresses <|turn>model after tool responses, which prevents the model from re-entering its thinking state in multi-turn tool-calling flows.

Fix:
- Remove 'tool_response' from the suppression condition so the model gets its turn marker after tool responses.
- Inject <|channel>thought when enable_thinking=true AND the previous message was a tool_response, nudging the model back into its reasoning state machine.

Normal (non-tool) turns are unchanged — the model self-generates the thinking channel cue as usual.

Ref: https://github.com/vllm-project/vllm/issues/45039

Files changed (1) hide show
  1. chat_template.jinja +4 -1
chat_template.jinja CHANGED
@@ -374,8 +374,11 @@
374
  {%- endfor -%}
375
 
376
  {%- if add_generation_prompt -%}
377
- {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
378
  {{- '<|turn>model\n' -}}
 
 
 
379
  {%- endif -%}
380
 
381
  {%- if not enable_thinking -%}
 
374
  {%- endfor -%}
375
 
376
  {%- if add_generation_prompt -%}
377
+ {%- if ns.prev_message_type != 'tool_call' -%}
378
  {{- '<|turn>model\n' -}}
379
+ {%- if enable_thinking and ns.prev_message_type == 'tool_response' -%}
380
+ {{- '<|channel>thought\n' -}}
381
+ {%- endif -%}
382
  {%- endif -%}
383
 
384
  {%- if not enable_thinking -%}