OMLX.md

Using This Quant With OMLX

Some OMLX builds may not yet include native Solar Open2 support in their bundled mlx_lm runtime. This repo includes a small compatibility patch that installs:

• solar_open2.py, the MLX loader for the Solar Open2 architecture.
• solar_open2_tool_parser.py, a parser for Solar Open2 tool-call markup.
• Tokenizer runtime detection for Solar Open2 thinking and tool-call markers.
• Four-state recurrent-cache restoration for reliable multi-turn tool loops.

Install

Clone this model repo locally, then run the installer from the repo root:

sudo zsh omlx/install_omlx_solar_open2_patch.sh

The installer patches the OMLX app bundle and creates timestamped backups of any files it changes. If your OMLX installation uses a non-default location, set:

export OMLX_MLX_LM_DIR="/path/to/mlx_lm"
export OMLX_PYTHON="/path/to/python"
export OMLX_RESOURCES="/path/to/oMLX.app/Contents/Resources"
sudo -E zsh omlx/install_omlx_solar_open2_patch.sh

Reload OMLX after installing the patch.

Tool Calling

Solar Open2 emits tool calls using this marker format:

<|tool_call:start|>tool_name
<|tool_arg:start|>argument_name<|tool_arg:value|>argument_value<|tool_arg:end|>
<|tool_call:end|>

The included parser converts that format into OpenAI-compatible tool-call arguments where the serving runtime supports tool parsing.

Thinking Toggle

This quant defaults to clean direct responses. To enable or disable thinking in clients that pass chat-template arguments:

{"enable_thinking": false}

{"enable_thinking": true}

Advanced clients can also pass Solar's native option directly:

{"reasoning_effort": "none"}

{"reasoning_effort": "high"}

If the serving runtime does not hide reasoning channels, enabled thinking may show <|think:start|> and <|think:end|> markers in generated text.

Notes

OMLX updates can replace bundled runtime files. Re-run the installer after an OMLX update if the model stops loading or tool-call parsing disappears.