OliviaRossi/Improved-Chat-Template-for-Qwen-3.x

Improved Chat Template for Qwen 3.x

An advanced, production-hardened Jinja2 Chat Template engineered specifically for Qwen 2.5, Qwen 3.x, QwQ, and Hybrid Reasoning / Tool-Calling LLMs.

This template delivers a resilient, unified prompt engine that seamlessly integrates multi-tier Chain-of-Thought (CoT) reasoning, dual-format agentic tool execution, automatic error-recovery heuristics, and strict token-waste elimination.

---

🏛️ Lineage & Acknowledgments

This template is a synthesis and evolution built directly upon architectural breakthroughs and design patterns from four foundational templates in the open-source community:

1. Jackrong/Qwen-Tool-Calling-Template-Fix:
   • Contributed the core foundation for multi-system message head aggregation, inline control token stripping, multi-turn synthetic agent loop index recovery, and the consecutive tool error classification warning loop.
2. peculiar-ragdoll/Qwen-Sharp-Chat-Templates:
   • Contributed dynamic thinking-aware tool prompt adapting (preventing tool prompt <think> instructions when reasoning is disabled), optional conciseness (terse) system prompt controls, and broad message.reasoning / message.thinking field extraction.
3. Kwaipilot/KAT-Coder-V2.5-Dev:
   • Contributed robust multimodal vision pad counting macros (<|vision_start|><|image_pad|><|vision_end|>), vision parameter indexing, and clean function call parameter serialization.
4. ornith-ai/Ornith-1.5-35B-A3B:
   • Contributed baseline XML parameter structuring mechanisms, strict role separation checks, and early prompt validation concepts.

---

⚡ Key Improvements & Innovations in v22.5.1

• 🧠 Granular Reasoning Effort & Dynamic Think Control: Control thinking depth on-the-fly via API kwargs (reasoning_effort="low"|"medium"|"xhigh"|"ultracode"|"extreme"|"deep"|"long") or inline prompt triggers (<|think_off|>, <|think_xhigh|>, <|think_deep|>).
• 🛠️ Dual-Mode Agentic Tool Calling: Seamlessly toggle between native Qwen XML format (<tool_call><function=...></tool_call>) and OpenAI-compatible JSON format ({"name": ...}) without prompt drift.
• 🛡️ Self-Healing Tool Heuristics: Automatically detects tool execution failures (non-zero exit codes, Python tracebacks, HTTP 404/500, Rate Limits, connection drops) and injects progressive diagnostic warnings (⚠️ SYSTEM WARNING) into subsequent model turns.
• 🧹 Zero Ghost-Think Bloat: Historical turns without reasoning do not output empty <think></think> blocks into context, preserving precious KV cache space.
• 🔒 Crash-Proof Multi-Turn Text Preservation: Safe multi-split logic prevents text drops when assistants write markdown or code blocks containing </think> tags.
• 🌐 API Gateway & Proxy Resilient: Built-in normalization for string-booleans ("false", '0') and safe integer casting to prevent Python runtime TypeError crashes in FastAPI, vLLM, LiteLLM, and OpenRouter environments.

---

📊 Feature Comparison Matrix

Feature	Stock Qwen Template	Strict Terse Templates	Jack Template (v22.5.1)
**Inline Think Toggles (`<	think_*	>`)**	❌ No
History Reasoning Sanitization	❌ Injects Empty Tags	⚠️ Inconsistent	✅ Smart (No Ghost Tags)
Consecutive Tool Error Warning Loop	❌ No	❌ No	✅ Dynamic 2-Stage Warning
Dual Tool Modes (XML & JSON)	⚠️ XML only	⚠️ Inflexible	✅ XML + JSON Adaptive
String-Boolean & API Type Safety	❌ Crashes on "false"	❌ Crashes	✅ Fully Normalized
Reasoning Preservation (reasoning_content)	⚠️ Flaky	⚠️ Drops dicts	✅ Strings & Dicts Supported
Unconstrained Deep Reasoning by Default	✅ Yes	❌ Forced terseness	✅ Full Performance Preserved

---

⚙️ Configurable Template Parameters (chat_template_kwargs)

All parameters are optional and can be supplied via chat_template_kwargs or API request bodies:

Parameter	Type	Default	Description
enable_thinking	bool / str	true	Enables or disables the <think> reasoning block. Accepts true, false, "true", "false", 1, 0.
reasoning_effort	str	"medium"	Sets reasoning intensity: 'none', 'low', 'minimal', 'medium', 'high', 'xhigh', 'ultracode', 'extreme', 'deep', 'long'.
tool_call_format	str	'xml'	Chooses between 'xml' (Qwen native <function=...>) and 'json' ({"name": ...}).
auto_disable_thinking_with_tools	bool / str	false	Automatically disables reasoning during tool-calling turns for low-latency agent routing.
max_tool_arg_chars	int / str	0 (disabled)	Truncates tool argument strings longer than $N$ characters to prevent memory/context overflows.
max_tool_response_chars	int / str	0 (disabled)	Truncates tool execution outputs longer than $N$ characters before injection into context.
preserve_reasoning	bool / str	true	Preserves historical thinking blocks across multi-turn chats. Set false to drop past reasoning while retaining current turn reasoning.
add_vision_id	bool / str	false	Prepends human-readable labels (Picture 1: , Video 1: ) before multimodal vision pads.
terse	bool / str	false	Enables strict conciseness prompt rules. Kept false by default to avoid harming deep CoT reasoning.

---

🎯 Inline Reasoning Control Tokens

You can dynamically adjust reasoning on a per-turn basis by including any of the following tags anywhere inside user or system messages:

• Disable Thinking: <|think_off|>
• Enable Standard Thinking: <|think_on|>, <|think_medium|>
• Low / Fast Thinking: <|think_low|>, <|think_minimal|>
• Deep / Ultracode Thinking: <|think_high|>, <|think_xhigh|>, <|think_max|>, <|think_ultracode|>, <|think_extreme|>, <|think_deep|>, <|think_long|>

Note: The template automatically strips these control tokens during rendering, preventing token leakage into the final context.

---

🚀 Deployment & Integration

1. Hugging Face Transformers (Python)

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-72B-Instruct")

# Load template from file
with open("chat_template.jinja", "r") as f:
    tokenizer.chat_template = f.read()

messages = [
    {"role": "system", "content": "You are an expert systems programmer."},
    {"role": "user", "content": "<|think_xhigh|> Implement a lock-free ring buffer in C++."}
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    chat_template_kwargs={
        "enable_thinking": True,
        "reasoning_effort": "xhigh"
    }
)
print(prompt)

---

2. vLLM Server Launch

Pass the template directly to the vLLM OpenAI-compatible server:

vllm serve Qwen/Qwen2.5-32B-Instruct \
    --chat-template ./chat_template.jinja \
    --enable-auto-tool-choice \
    --tool-call-parser hermes

---

3. SGLang Server Launch

python3 -m sglang.launch_server \
    --model-path Qwen/Qwen2.5-32B-Instruct \
    --chat-template ./chat_template.jinja \
    --port 30000

---

4. LiteLLM / Proxy Configuration

model_list:
  - model_name: qwen-jack
    litellm_params:
      model: vllm/Qwen/Qwen2.5-72B-Instruct
      api_base: http://localhost:8000/v1
      chat_template: ./chat_template.jinja

---

📄 License

This project is licensed under the Apache License 2.0. You are free to use, modify, distribute, and integrate it into commercial and open-source systems.