Instructions to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16 # Run inference directly in the terminal: ./llama-cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Use Docker
docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
- LM Studio
- Jan
- vLLM
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
- Ollama
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Ollama:
ollama run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
- Unsloth Desktop
- Pi
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Docker Model Runner:
docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
- Lemonade
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Run and chat with the model
lemonade run user.Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF-BF16
List all available models
lemonade list
- Hermes Agent
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Polish README wording and clarify runtime distinctions
Browse files
README.md
CHANGED
|
@@ -84,7 +84,7 @@ This model is designed as a **coding agent model** for use with Claude Code, Ope
|
|
| 84 |
|
| 85 |
## Why IQ4_NL is Recommended Over BF16, Q8, and Q6
|
| 86 |
|
| 87 |
-
This is not typical.
|
| 88 |
|
| 89 |
1. **Quantization acts as a regularizer** — slight weight rounding prevents degenerate thinking loops that BF16 and Q6_K are prone to
|
| 90 |
2. **Custom imatrix preserves reasoning weights** — 30% reasoning/self-verification calibration data ensures chain-of-thought and self-correction weights are preserved
|
|
@@ -106,7 +106,7 @@ The IQ4_NL with custom imatrix was the **only variant that produced working data
|
|
| 106 |
|
| 107 |
**Supporting evidence: 5 automated coding problems (merge intervals, LCS, RPN evaluator, valid parentheses, trapping rain water) tested at both temperature 0.6 and 0.8 — IQ4_NL with custom imatrix scored 5/5 at both temperatures.** BF16 and Q6_K both entered infinite thinking loops on the same problems.
|
| 108 |
|
| 109 |
-
For clarity: the results above come from the **ik-llama.cpp-oriented IQ4_NL build**, which remains the author's primary recommendation. A separate standard llama.cpp-compatible IQ4_NL build is
|
| 110 |
|
| 111 |
## Critical: KV Cache Recommendations
|
| 112 |
|
|
@@ -197,12 +197,23 @@ The resulting standard llama.cpp-compatible file still uses a mixed quant layout
|
|
| 197 |
|
| 198 |
The author still personally recommends the **ik-llama** build because that is the day-to-day driver and the one that will be exercised the most in live coding / agent workloads.
|
| 199 |
|
|
|
|
|
|
|
| 200 |
## Recommended Parameters
|
| 201 |
|
| 202 |
These are the **current best test parameters** for this repo so far:
|
| 203 |
|
| 204 |
```bash
|
| 205 |
-
llama-server
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 206 |
```
|
| 207 |
|
| 208 |
**Current params:**
|
|
@@ -274,7 +285,7 @@ Primary validation for this release used **ik-llama.cpp**, mainly because its **
|
|
| 274 |
| Flash Attention | ON (`GGML_CUDA_FA_ALL_QUANTS`) |
|
| 275 |
| Build Date | Apr 5, 2026 |
|
| 276 |
|
| 277 |
-
The author uses ik-llama because graph split performs much better on multi-GPU setups.
|
| 278 |
|
| 279 |
### standard llama.cpp (used for the new compatibility build)
|
| 280 |
|
|
@@ -299,14 +310,14 @@ llama-quantize \
|
|
| 299 |
IQ4_NL
|
| 300 |
```
|
| 301 |
|
| 302 |
-
Compatibility
|
| 303 |
|
| 304 |
- `CUDA_VISIBLE_DEVICES=0,1,2`
|
| 305 |
- `--tensor-split 3,2,2`
|
| 306 |
- `-c 262144`
|
| 307 |
- `--cache-type-k f32 --cache-type-v f32`
|
| 308 |
|
| 309 |
-
Verification prompts returned:
|
| 310 |
|
| 311 |
- `0.05`
|
| 312 |
- `5050`
|
|
|
|
| 84 |
|
| 85 |
## Why IQ4_NL is Recommended Over BF16, Q8, and Q6
|
| 86 |
|
| 87 |
+
This is not typical. In the author's tested coding / agent workloads, the **preferred IQ4_NL quantization with custom importance matrix consistently outperformed Q8_0, Q6_K, and full-precision BF16**:
|
| 88 |
|
| 89 |
1. **Quantization acts as a regularizer** — slight weight rounding prevents degenerate thinking loops that BF16 and Q6_K are prone to
|
| 90 |
2. **Custom imatrix preserves reasoning weights** — 30% reasoning/self-verification calibration data ensures chain-of-thought and self-correction weights are preserved
|
|
|
|
| 106 |
|
| 107 |
**Supporting evidence: 5 automated coding problems (merge intervals, LCS, RPN evaluator, valid parentheses, trapping rain water) tested at both temperature 0.6 and 0.8 — IQ4_NL with custom imatrix scored 5/5 at both temperatures.** BF16 and Q6_K both entered infinite thinking loops on the same problems.
|
| 108 |
|
| 109 |
+
For clarity: the results above come from the **ik-llama.cpp-oriented IQ4_NL build**, which remains the author's primary recommendation. A separate standard llama.cpp-compatible IQ4_NL build is provided for portability.
|
| 110 |
|
| 111 |
## Critical: KV Cache Recommendations
|
| 112 |
|
|
|
|
| 197 |
|
| 198 |
The author still personally recommends the **ik-llama** build because that is the day-to-day driver and the one that will be exercised the most in live coding / agent workloads.
|
| 199 |
|
| 200 |
+
Both IQ4_NL releases are mixed quants. The key difference is that the `ik-llama` build uses ik-specific tensor types, while the `llama.cpp-compatible` build uses mainline-supported tensor types.
|
| 201 |
+
|
| 202 |
## Recommended Parameters
|
| 203 |
|
| 204 |
These are the **current best test parameters** for this repo so far:
|
| 205 |
|
| 206 |
```bash
|
| 207 |
+
llama-server \
|
| 208 |
+
-m RYS-Qwen3.5-27B-Uncensored-Splice-IQ4_NL-ik-llama.gguf \
|
| 209 |
+
--host 0.0.0.0 --port 8001 \
|
| 210 |
+
-ngl 99 -c 262144 \
|
| 211 |
+
--cache-type-k f16 --cache-type-v f16 \
|
| 212 |
+
--cache-ram 30720 \
|
| 213 |
+
--flash-attn on \
|
| 214 |
+
--jinja --reasoning-format deepseek \
|
| 215 |
+
--temp 0.7 --top-p 0.95 --top-k 20 \
|
| 216 |
+
--min-p 0.0 --repeat-penalty 1.0
|
| 217 |
```
|
| 218 |
|
| 219 |
**Current params:**
|
|
|
|
| 285 |
| Flash Attention | ON (`GGML_CUDA_FA_ALL_QUANTS`) |
|
| 286 |
| Build Date | Apr 5, 2026 |
|
| 287 |
|
| 288 |
+
The author uses ik-llama because graph split performs much better on multi-GPU setups. It remains the author's **personal daily driver** and the variant that will receive the most testing.
|
| 289 |
|
| 290 |
### standard llama.cpp (used for the new compatibility build)
|
| 291 |
|
|
|
|
| 310 |
IQ4_NL
|
| 311 |
```
|
| 312 |
|
| 313 |
+
Compatibility validation used:
|
| 314 |
|
| 315 |
- `CUDA_VISIBLE_DEVICES=0,1,2`
|
| 316 |
- `--tensor-split 3,2,2`
|
| 317 |
- `-c 262144`
|
| 318 |
- `--cache-type-k f32 --cache-type-v f32`
|
| 319 |
|
| 320 |
+
Verification prompts returned correct final outputs:
|
| 321 |
|
| 322 |
- `0.05`
|
| 323 |
- `5050`
|