cyankiwi

/

gemma-4-26B-A4B-it-AWQ-4bit

Like 98

Follow

cyankiwi 830

chat_template.jinja in this repo is byte-identical to upstream google/gemma-4-26B-A4B-it at revision 4c55b528bd (PR #36, 2026-04-28) and is two template fixes behind. This PR syncs it with upstream main.

Missing upstream fixes

Concrete breakage from the missing #47

Any OpenAI-compatible client that sends a role: tool message whose tool_call_id does not resolve against the preceding assistant tool_calls gets an HTTP 400 from vLLM:

TypeError: can only concatenate str (not "NoneType") to str
  in safe_apply_chat_template -> chat_template.jinja
{'error': {'message': 'can only concatenate str (not "NoneType") to str',
           'type': 'BadRequestError', 'param': None, 'code': 400}}

The template resolves the function name only via tool_call_id -> tool_calls[].id, and the intended fallback does not fire:

{%- set ns_tname = namespace(name=follow.get('name') | default('unknown')) -%}

Jinja's default filter only replaces undefined values, not Python None. dict.get() returns None when the key is absent, so tool_name stays None and 'response:' + tool_name raises. Upstream #47 fixes exactly this (or 'unknown', and | default('unknown', true) at the second call site).

Minimal reproduction, no GPU needed:

from jinja2.sandbox import ImmutableSandboxedEnvironment
env = ImmutableSandboxedEnvironment(trim_blocks=True, lstrip_blocks=True)
env.globals["raise_exception"] = lambda m: (_ for _ in ()).throw(RuntimeError(m))
tpl = env.from_string(open("chat_template.jinja").read())

tpl.render(
    messages=[
        {"role": "user", "content": "turn off the light"},
        {"role": "assistant", "content": None, "tool_calls": [
            {"id": "call_1", "type": "function",
             "function": {"name": "turn_off", "arguments": {"name": "light"}}}]},
        {"role": "tool", "tool_call_id": "does_not_match", "content": "{}"},
    ],
    tools=[], bos_token="<bos>", add_generation_prompt=True, enable_thinking=False,
)
# current file: TypeError: can only concatenate str (not "NoneType") to str
# upstream main: renders fine, falls back to `response:unknown{...}`

In the wild this is reached through clients that inject synthetic tool messages. One example is the Home Assistant integration skye-harris/hass_local_openai_llm, which injects context as a role: tool message with a fabricated tool_call_id; that is arguably a client-side bug, but the template's own fallback is meant to handle it and currently does not.

Possibly related: #2 ("Use in OpenCode via vLLM: Endless Tool Calling Loops & undesirable behavior"). Upstream #47's changelog lists turn-tag balance, and participants in that thread already worked around it by swapping in a different Jinja template.

Why this is safe for a quantized repo

• tokenizer.json here is byte-identical to upstream (sha256 cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f), so the vocabulary and special tokens match exactly. No requantization, no weight changes.
• Rendering is unchanged for well-formed conversations. I diffed the rendered output of the current file against upstream main across a range of message shapes (plain user turn, open tool call, completed tool round-trip, multi-turn with tool history, two consecutive round-trips). With enable_thinking: false the output is byte-identical in every case. With enable_thinking: true there is exactly one difference: after a tool response, upstream opens the thought channel in the generation prompt (<|channel>thought\n) where the current file emits nothing — that is #47's intended reasoning-preservation fix, not a regression. Beyond that, only the previously-crashing shapes change behaviour.
• Verified against vLLM 0.27.1 with --tool-call-parser gemma4 --reasoning-parser gemma4: tool calling works end to end, and the 400 is gone.

One caveat worth flagging for anyone adopting upstream main: it now hard-rejects tool_calls[].function.arguments passed as a JSON string via raise_exception. That is fine under vLLM, whose _postprocess_messages() parses the string into a dict before rendering, but a client calling apply_chat_template() directly with raw OpenAI-format messages would need to deserialize first.

Not included

tokenizer_config.json is also behind upstream: it lacks the response_template key added in 4d7ae4984b (#50, 2026-07-20). That is the only difference between the two files. I left it out to keep this diff focused — happy to add it if you would like both synced in one go.

File taken verbatim from google/gemma-4-26B-A4B-it at main (4d7ae4984b), sha256 ae53464bf3be25802b3a5b37def7fd89667067d7577049b3b2d74c4d8de4c6d4.