poolside

/

Laguna-S-2.1-NVFP4

like 88

Follow

Poolside 815

Openrouter version is doing this too. Model unusable

Is this expected from the NVFP4 quant? I hope it's either isolated, or an easily resolved issue, because when it works, it works.

Seems to be happening for all quantizations + for Poolside's own official API, not sure what's going on.

Without --default-chat-template-kwargs '{"enable_thinking": true}'

Nvpf4 ok for about 4 hours, similar sessions by orchestration ( deep dive audit complex apps one by one) without vllm 'enable thinking ' flag. Laguna suggested some good findings that qwen3.5 122b or nemotron 3 super didn't mentioned.
In case not with orchestration, few loops , not repeated but kind like: after it refactoring large html file, it found extra at line 3550 then repeatedly: i need to look from line 3335 to 3550; look from 3345 to 3550 ... Increase/reduce starting line by 10 each tool call 😂

With
--default-chat-template-kwargs '{"enable_thinking": true}'

After enabled thinking in vllm. Loops all the time with xhigh effort, restarted vllm asked to retry still loop at same synced task 😁 . Just turn off vllm flag only, restart vllm , keep harness session with xhigh effort then asked to retry, this time passed and is moving forward.

Maybe something with reasoning cap that cause serious loops. Just guessing .

Update: gg ai mode advice

Here is the professional, technical response tailored for your reply on the Hugging Face discussion thread:

Yes, exactly! I removed the --default-chat-template-kwargs flag from the vLLM server entirely and let the orchestration harness/scaffold dynamically handle the reasoning requests.
Here is a breakdown of the technical root causes behind why this fixes the issue:

1. Asymmetric Pipeline Overlap & Context Purging

When --default-chat-template-kwargs '{"enable_thinking": true}' is enforced at the vLLM server level, it injects reasoning tokens globally across the entire session generation. When an agent orchestration framework executes continuous multi-turn tool calls, these persistent internal thinking sequences overload the context history.
By offloading the reasoning control to the client-side harness, the framework dynamically requests thinking tokens only during heavy cognitive tasks and strips out the verbose internal reasoning metadata before appending the final action to the execution history. This keeps the vLLM KV Cache clean and prevents context bloating.

2. Attention Weight Atrophy & Convergence Failure

The loop where the model increments/decrements lines step-by-step (e.g., drifting from line 3335 to 3345 while looking for line 3550) is a mathematical stagnation artifact.
When the context window is flooded with static reasoning tokens from previous system steps, the attention weights distributed to absolute anchors (like static line numbers or file offsets) decay severely. The model suffers from a convergence failure in its next-token probability distribution. It cannot pinpoint the exact structural location anymore, causing the autoregressive decoder to generate linearly shifting offsets (e.g., sliding by exactly 10 units each tool call) in a desperate attempt to find the anchor.

3. Reasoning Boundary Collapse during Tool Interactivity

Enforcing static server-side thinking causes a structural collapse between the model's internal scratchpad tokens (<|thought|>) and its functional payload tokens (Tool Parameters). The model loses track of its operational constraints and hallucinates localized patch sequences or loops state checks on identical blocks.
Omit the server-side parameter to allow the API server to perform dynamic parsing only when explicitly triggered by client payloads. This restores execution stability for long agentic workflows on local setups.

After enabled thinking in vllm. Loops all the time with xhigh effort, restarted vllm asked to retry still loop at same synced task 😁 . Just turn off vllm flag only, restart vllm , keep harness session with xhigh effort then asked to retry, this time passed and is moving forward.

Do you mean you removed the enable_thinking from the vllm command, but request thinking from the harness/scaffold?

Yes, removed the enable_thinking from the vllm command, but request thinking from the harness/scaffold
my agent is copilot cli , with reasoning effort xhigh for 3 parallel audit sessions.

• however new issue, vllm just suddenly idle for more than 10', when 1 finished and two were conducting report. I needed to restart and now with copilot effort high to see

This model not stable, what I do is to evaluate/ benchmark it against 122b qwen nvpf4 in my real repo for code reviews.
So far it's findings are better, just don't know when it crashed/loop.