# Using This Quant With OMLX

Some OMLX builds may not yet include native Solar Open2 support in their bundled
`mlx_lm` runtime. This repo includes a small compatibility patch that installs:

- `solar_open2.py`, the MLX loader for the Solar Open2 architecture.
- `solar_open2_tool_parser.py`, a parser for Solar Open2 tool-call markup.
- Tokenizer runtime detection for Solar Open2 thinking and tool-call markers.
- Four-state recurrent-cache restoration for reliable multi-turn tool loops.

## Install

Clone this model repo locally, then run the installer from the repo root:

```bash
sudo zsh omlx/install_omlx_solar_open2_patch.sh
```

The installer patches the OMLX app bundle and creates timestamped backups of any
files it changes. If your OMLX installation uses a non-default location, set:

```bash
export OMLX_MLX_LM_DIR="/path/to/mlx_lm"
export OMLX_PYTHON="/path/to/python"
export OMLX_RESOURCES="/path/to/oMLX.app/Contents/Resources"
sudo -E zsh omlx/install_omlx_solar_open2_patch.sh
```

Reload OMLX after installing the patch.

## Tool Calling

Solar Open2 emits tool calls using this marker format:

```text
<|tool_call:start|>tool_name
<|tool_arg:start|>argument_name<|tool_arg:value|>argument_value<|tool_arg:end|>
<|tool_call:end|>
```

The included parser converts that format into OpenAI-compatible tool-call
arguments where the serving runtime supports tool parsing.

## Thinking Toggle

This quant defaults to clean direct responses. To enable or disable thinking in
clients that pass chat-template arguments:

```json
{"enable_thinking": false}
```

```json
{"enable_thinking": true}
```

Advanced clients can also pass Solar's native option directly:

```json
{"reasoning_effort": "none"}
```

```json
{"reasoning_effort": "high"}
```

If the serving runtime does not hide reasoning channels, enabled thinking may
show `<|think:start|>` and `<|think:end|>` markers in generated text.

## Notes

OMLX updates can replace bundled runtime files. Re-run the installer after an
OMLX update if the model stops loading or tool-call parsing disappears.