deepreinforce-ai

/

Ornith-1.0-35B-GGUF

like 640

Follow

DeepReinforce 1.3k

In what sense its bad, its quite good punching above its weight, its working out good for me with my own agent setup.

Interesting, for me it just failed tool calls.

Generated without returning any responses until llama.cpp llama-server was stopped (GPU utilization high, no responses, multiple minutes).

Probably related to this https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/discussions/10

Just in case, maybe copy of the Qwen 3.6 parameters is wrong?
On reddit there are comments that reasoning should be disabled for agent setup, maybe then it'll work - but I've Qwen 3.6 with reasoning enabled working with the same coding agent.

./bin/llama-server \
--port 5848 \
--jinja \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.0 \
--reasoning on \
--spec-type ngram-mod \
--no-mmap \
--flash-attn on \
--fit on \
--fit-ctx 262144 \
--fit-target 2048 \
--no-mmproj-offload \
--chat-template-kwargs '{"preserve_thinking": true}' \
--model ./models/ornith-1.0-35b-Q6_K.gguf \
--mmproj ./models/mmproj-Qwen_Qwen3.6-35B-A3B-bf16.gguf

great TTFT tho...

Came here to defend. To come and throw such a bold claim in a sophisticated community, it's at least nice of you to include use cases, your setup, and why you think it's 'terrible'. You're on the 35B page btw, not the 9B one.

Your experience depends on A LOT of things. Either share more, or we thank you for your opinion, but take it lightly. 🤗

Interesting, for me it just failed tool calls.

Generated without returning any responses until llama.cpp llama-server was stopped (GPU utilization high, no responses, multiple minutes).

Probably related to this https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/discussions/10

Just in case, maybe copy of the Qwen 3.6 parameters is wrong?
On reddit there are comments that reasoning should be disabled for agent setup, maybe then it'll work - but I've Qwen 3.6 with reasoning enabled working with the same coding agent.

./bin/llama-server \
--port 5848 \
--jinja \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.0 \
--reasoning on \
--spec-type ngram-mod \
--no-mmap \
--flash-attn on \
--fit on \
--fit-ctx 262144 \
--fit-target 2048 \
--no-mmproj-offload \
--chat-template-kwargs '{"preserve_thinking": true}' \
--model ./models/ornith-1.0-35b-Q6_K.gguf \
--mmproj ./models/mmproj-Qwen_Qwen3.6-35B-A3B-bf16.gguf

I don't think that this model has an mmproj attached, and I also don't think it has speculative decoding. These are the first two parameters I'd suspect. I use this and it works flawlessly in OpenCode:

llama-server.exe -m E:/1models/DeepReinforce/ornith-1.0-35b-Q6_K.gguf --ctx-size 131072 -ctk q4_0 -ctv q4_0 -fa on -ngl -1 --main-gpu 1 -t 8 -b 2048 -ub 2048 --host 127.0.0.1 --port 8081 --no-mmap --mlock --temp 0.6 --top-p 0.95 --top-k 20 --cache-reuse 256 --jinja --fit on

I get around ~400t/s pp and ~40t/s tg on 5070 Ti and and 64GB DDR4 RAM.

I get around ~400t/s pp and ~40t/s tg on 5070 Ti and and 64GB DDR4 RAM.

Yes, performance is great up until the tool call for me using late-cli. E.g. from the Web UI with tool calls disabled.

don't think it has speculative decoding

As far as I know ngram cache variants (including ngram-mod) do not need any model side speculative decoding support (no MTP layers, no separate draft models required).
Server side caching of the token sequences that are already generated and are retrieved from the cache if matched: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md#n-gram-cache-ngram-cache.

I don't think that this model has an mmproj attached

Yes, but as this model is Qwen3.5-35B-A3B finetune I just tried Qwen3.6-35B-A3B mmproj from bartowski/Qwen_Qwen3.6-35B-A3B-GGUF.
Works perfectly in server's Web UI, for example with this image https://en.wikipedia.org/wiki/Computer_network_diagram#/media/File:Sample-network-diagram.png and prompt "Describe this image":

This image is a simple, black-and-white network topology diagram
illustrating how different devices are connected to form a local
area network (LAN) that accesses the internet.

Here is a breakdown of the components from left to right:

• Client Computers: On the far left, there are three desktop
  computer setups (each consisting of a tower and a monitor).
  They are arranged with two at the top and one at the bottom.
• Switch: In the center-left, a grey box labeled "Switch"
  acts as the central hub. All three client computers are connected to it via lines.
• Server: Connected to the right side of the switch is a tall,
  rack-mounted unit labeled "Server". It appears to have multiple drive bays or slots.
• Printer: Directly above the server is an icon of a printer labeled "Printer".
  A line connects it down into the top of the server, indicating it is likely a network printer hosted by the server.
• Router: To the right of the server is a box labeled "Router",
  depicted with arrows pointing in four directions. It is connected to the server.
• The Internet: At the top right, a cloud shape labeled "The Internet"
  is connected to the router by a jagged lightning bolt line, representing the connection to the wider web.

Summary of Flow: The diagram shows data flowing from the three client
PCs through the Switch, into the Server (which also manages the Printer),
out through the Router, and finally to The Internet.