ThrBil commited on
Commit
c75415f
·
verified ·
1 Parent(s): d1fef18

chat_template: replace no-user-query raise_exception with tolerant fallback

Browse files

The multi_step_tool guard hard-raises when a conversation contains no plain user message (shape: [system, assistant, tool, ...]). Agent frameworks produce exactly this shape after history compression, and runtimes that render the template during automatic tool-parser generation (vLLM >= 0.24, llama.cpp, LM Studio, Ollama) fail the entire request with a 400 before generating a token — see vllm-project/vllm#36432 and Qwen/Qwen3.6-27B discussion #16 for the same failure on sibling models.

ns.last_query_index only gates reasoning-content preservation, so the safe fallback is -1 (treat every assistant turn as after the last query, i.e. preserve reasoning). Verified locally on vLLM 0.24: the failing shape renders and generates correctly with this template, and normal chat / multi-step tool conversations are byte-identical.

Files changed (1) hide show
  1. chat_template.jinja +6 -1
chat_template.jinja CHANGED
@@ -76,7 +76,12 @@
76
  {%- endif %}
77
  {%- endfor %}
78
  {%- if ns.multi_step_tool %}
79
- {{- raise_exception('No user query found in messages.') }}
 
 
 
 
 
80
  {%- endif %}
81
  {%- for message in messages %}
82
  {%- set content = render_content(message.content, true)|trim %}
 
76
  {%- endif %}
77
  {%- endfor %}
78
  {%- if ns.multi_step_tool %}
79
+ {#- Tolerate conversations with no plain user turn (e.g. [system, assistant, tool, ...],
80
+ common in agent frameworks after history compression). The hard raise breaks every
81
+ runtime that renders the template at tool-parser-generation time (vLLM >=0.24,
82
+ llama.cpp, LM Studio, Ollama): the request 400s before a single token is generated.
83
+ -1 == "no user query" == preserve reasoning content on all assistant turns. #}
84
+ {%- set ns.last_query_index = -1 %}
85
  {%- endif %}
86
  {%- for message in messages %}
87
  {%- set content = render_content(message.content, true)|trim %}