{#- ======== Template Parameters ======== #} {%- set add_generation_prompt = add_generation_prompt if add_generation_prompt is defined else true %} {%- set provider_system_prompt = provider_system_prompt if provider_system_prompt else false %} {%- set reasoning_effort = reasoning_effort if reasoning_effort is defined else "high" %} {%- set think_render_option = think_render_option if think_render_option is defined else "interleaved" %} {#- ======== System Block State ======== #} {%- set sys_ns = namespace(is_first_block=true) -%} {#- ======== Find last user message index ======== #} {%- set last_user_idx = namespace(value=-1) -%} {%- for message in messages -%} {%- if message.role == 'user' -%} {%- set last_user_idx.value = loop.index0 -%} {%- endif -%} {%- endfor -%} {#- ======== System messages renderers ======== #} {%- macro render_system_message(user_system_messages) %} {%- if provider_system_prompt %} {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %} {%- set sys_ns.is_first_block = false %} {{- "## Provider System Prompt\n\nYou are Solar Open2 250B, a large language model trained by Upstage AI, a Korean startup. Your knowledge cutoff is 2026-02. The current date is " + strftime_now("%Y-%m-%d") + "." }} {%- endif -%} {%- if user_system_messages %} {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %} {%- set sys_ns.is_first_block = false %} {{- "## System Prompt" }} {%- for system_message in user_system_messages %} {{- "\n\n" }} {{- system_message }} {%- endfor %} {%- endif -%} {%- endmacro %} {%- macro render_tool_instruction(tools, is_last_block) %} {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %} {%- set sys_ns.is_first_block = false %} {{- "## Tools\n- You may invoke one or more tools to assist with the user's query." }} {{- "\n\n### Available Tools\n" }} {%- for tool in tools %} {{- "<|tool:start|>" }} {{- tool.function | tojson }} {{- "<|tool:end|>\n" }} {%- endfor %} {{- "\n### Tool Call Instruction\n" }} {{- "- If using a tool, any reasoning must strictly precede the call. Do not append any text after the tool call.\n" }} {{- "- If no tool is required, answer directly from your knowledge without ever mentioning the availability or absence of tools.\n" }} {{- "- Each tool call MUST use this following format: <|tool_call:start|>{example-tool-name}\n<|tool_arg:start|>{example-key-name-1}<|tool_arg:value|>{example-value-1}<|tool_arg:end|>\n<|tool_arg:start|>{example-key-name-2}<|tool_arg:value|>{example-value-2}<|tool_arg:end|>\n<|tool_call:end|>\n" }} {%- endmacro %} {#- Input convention for `response_format`: - Per the OpenAI Chat Completions API, `response_format` is an object of one of: {"type": "json_schema", "json_schema": {"name": ..., "schema": {...}, "strict": ...}} {"type": "json_object"} - Only `response_format.json_schema.schema` (the inner JSON Schema) is injected into the prompt; the OpenAI wrapper fields (`name`, `strict`, ...) are dropped. - For `json_object`, no schema exists, so we emit a generic JSON-only instruction. - vLLM does NOT pass `response_format` to chat templates by default; it routes it to guided decoding instead. This template/whale opts into prompt injection, so callers serving the model directly via vLLM must opt in via `chat_template_kwargs` (or rely on whale's normalization). -#} {%- macro render_json_response_format_instruction(response_format) %} {%- if not sys_ns.is_first_block %}{{- "\n\n" }}{%- endif %} {%- set sys_ns.is_first_block = false %} {{- "## Output Format Constraint" }} {%- if response_format.type == "json_schema" %} {{- "\n\n- Your final response should follow the JSON schema:\n```json\n" }} {{- response_format.json_schema.schema | tojson }} {{- "\n```\n- Please ensure your answers adhere to this format and do not contain any unnecessary text." }} {%- elif response_format.type == "json_object" %} {{- "\n\n- Your final response must be a valid JSON object." }} {{- "\n- Do not include any text outside the JSON object." }} {%- endif %} {%- endmacro %} {#- Input convention for `tool_arguments`: - Must be a mapping (dict), NOT a JSON-encoded string. - The OpenAI API spec defines `tool_calls[].function.arguments` as a JSON string, but vLLM, HuggingFace transformers tool-use templates (Qwen, Hermes, Llama, ...), and this template all expect the value to be parsed into a dict before rendering. - vLLM normalizes this in `_postprocess_messages()` (vllm/entrypoints/chat_utils.py). - Callers (whale, training pipelines that call `tokenizer.apply_chat_template` directly, ...) are responsible for parsing the JSON string into a dict before invoking the template. -#} {%- macro render_tool_arguments(tool_arguments) %} {%- for args_name, args_value in tool_arguments|items %} {{- "<|tool_arg:start|>"+ args_name + "<|tool_arg:value|>"}} {%- set args_value = args_value if args_value is string else (args_value | tojson | safe) %} {{- args_value }} {{- "<|tool_arg:end|>\n" }} {%- endfor %} {%- endmacro %} {#- ======== Render system message ======== #} {%- set ns = namespace(system_messages=[]) -%} {%- for message in messages -%} {%- if message.role == 'system' -%} {%- set ns.system_messages = ns.system_messages + [message.content] -%} {%- endif -%} {%- endfor -%} {%- if ns.system_messages or provider_system_prompt or tools or response_format -%} {{- "<|im:start|>system<|im:content|>" }} {{- render_system_message(ns.system_messages) }} {%- if tools -%} {{- render_tool_instruction(tools, not response_format) }} {%- endif %} {%- if response_format -%} {{- render_json_response_format_instruction(response_format) }} {%- endif %} {{- "<|im:end|>\n" }} {%- endif -%} {#- ======== Render main messages ======== #} {%- for message in messages -%} {%- if message.role == 'user' -%} {{- "<|im:start|>user<|im:content|>" + message.content + "<|im:end|>\n" }} {%- elif message.role == 'tool' -%} {%- set prev_is_tool = loop.index0 > 0 and messages[loop.index0 - 1].role == 'tool' -%} {#- Only render at the start of a contiguous tool block; subsequent tool messages -#} {#- are already emitted (reordered) within the first one. -#} {%- if not prev_is_tool -%} {%- set start_idx = loop.index0 -%} {#- Collect contiguous tool messages starting at start_idx -#} {%- set tool_ns = namespace(items=[], scanning=true) -%} {%- for j in range(start_idx, messages | length) -%} {%- if tool_ns.scanning -%} {%- if messages[j].role == 'tool' -%} {%- set tool_ns.items = tool_ns.items + [messages[j]] -%} {%- else -%} {%- set tool_ns.scanning = false -%} {%- endif -%} {%- endif -%} {%- endfor -%} {{- "<|im:start|>tool<|im:content|>" }} {%- set prev_msg = messages[start_idx - 1] if start_idx > 0 else none -%} {%- if prev_msg and prev_msg.role == 'assistant' and prev_msg.tool_calls -%} {#- Render tool responses in the order of the preceding assistant's tool_calls ids. -#} {%- set sep_ns = namespace(first=true, rendered_ids=[]) -%} {%- for tool_call in prev_msg.tool_calls -%} {%- for tool_msg in tool_ns.items -%} {%- if tool_msg.tool_call_id == tool_call.id -%} {%- if not sep_ns.first -%}{{- "\n" }}{%- endif -%} {%- set sep_ns.first = false -%} {%- set sep_ns.rendered_ids = sep_ns.rendered_ids + [tool_msg.tool_call_id] -%} {{- "<|tool_response:start|>" + tool_msg.content + "<|tool_response:end|>" }} {%- endif -%} {%- endfor -%} {%- endfor -%} {#- Append any orphan tool messages (no matching id) in their original order. -#} {%- for tool_msg in tool_ns.items -%} {%- if tool_msg.tool_call_id not in sep_ns.rendered_ids -%} {%- if not sep_ns.first -%}{{- "\n" }}{%- endif -%} {%- set sep_ns.first = false -%} {{- "<|tool_response:start|>" + tool_msg.content + "<|tool_response:end|>" }} {%- endif -%} {%- endfor -%} {%- else -%} {%- for tool_msg in tool_ns.items -%} {%- if not loop.first -%}{{- "\n" }}{%- endif -%} {{- "<|tool_response:start|>" + tool_msg.content + "<|tool_response:end|>" }} {%- endfor -%} {%- endif -%} {{- "\n<|im:end|>\n" }} {%- endif -%} {%- elif message.role == 'assistant' -%} {{- "<|im:start|>assistant<|im:content|>" }} {#- ======== Resolve reasoning field (reasoning or reasoning_content) ======== #} {%- if message.reasoning is defined -%} {%- set reasoning = message.reasoning -%} {%- elif message.reasoning_content is defined -%} {%- set reasoning = message.reasoning_content -%} {%- else -%} {%- set reasoning = none -%} {%- endif -%} {#- ======== Assistant Thinking ======== #} {%- if think_render_option == "preserved" -%} {%- if reasoning -%} {{- "<|think:start|>" + reasoning + "<|think:end|>" }} {%- else -%} {{- "<|think:start|><|think:end|>" }} {%- endif -%} {%- elif think_render_option == "interleaved" -%} {%- if reasoning and loop.index0 > last_user_idx.value -%} {{- "<|think:start|>" + reasoning + "<|think:end|>" }} {%- else -%} {{- "<|think:start|><|think:end|>" }} {%- endif -%} {%- endif -%} {#- ======== Assistant Messages ======== #} {%- if message.content -%} {{- message.content }} {%- endif -%} {#- ======== Assistant Tool calls ======== #} {%- if message.tool_calls -%} {%- for tool_call in message.tool_calls -%} {%- if not loop.first -%}{{- "\n" }}{%- endif -%} {{- "<|tool_call:start|>" + tool_call.function.name + "\n" }} {#- ======== FIXME: QWEN3.5 Method ======== #} {%- if tool_call.function.arguments is defined %} {{- render_tool_arguments(tool_call.function.arguments) }} {%- endif %} {{- "<|tool_call:end|>" }} {%- endfor -%} {{- "\n" }} {%- endif -%} {{- "<|im:end|>\n" }} {%- endif -%} {%- endfor -%} {%- if add_generation_prompt -%} {%- if reasoning_effort in ["medium", "high"] -%} {{- "<|im:start|>assistant<|im:content|><|think:start|>" }} {%- else -%} {{- "<|im:start|>assistant<|im:content|><|think:start|><|think:end|>" }} {%- endif -%} {%- endif -%}