Instructions to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Use Docker

docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

LM Studio
Jan

vLLM

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Ollama
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Ollama:
```
ollama run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
```
Unsloth Desktop

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Docker Model Runner:
```
docker model run hf.co/jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16
```

Lemonade

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Run and chat with the model

lemonade run user.Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jackasda211233/Qwen3.5-27B-Uncensored-RYS-Reasoner-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

jackasda211233 commited on Apr 8

Commit

b36b1f3

verified ·

1 Parent(s): db5c16c

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +10 -17

README.md CHANGED Viewed

@@ -82,28 +82,21 @@ This is not typical. Through extensive real-world testing, the **IQ4_NL quantiza
 2. **Custom imatrix preserves reasoning weights** — 30% reasoning/self-verification calibration data ensures chain-of-thought and self-correction weights are preserved
 3. **Half the size, better results** — 17GB vs 56GB, fits on a single 24GB GPU
-### Full Test Results
-**Test 1: Automated coding (5 LeetCode problems, auto-verified)**
-| Quant | imatrix | Temp 0.6 | Temp 0.8 |
-|-------|---------|----------|----------|
-| **IQ4_NL (custom)** | **reasoning** | **5/5** | **5/5** |
-| Q8_0 (custom) | reasoning | 5/5 | 5/5 |
-| Q6_K (any) | — | 2/5 (loops) | 1/5 (loops) |
-| BF16 | N/A | 2/5 (loops) | 2/5 (loops) |
-**Test 2: Full-stack project (AI Radar — 4 APIs, frontend, backend, DB, 256k context)**
-| Quant | imatrix | Completed | DB Works | Errors | Loops | Grade |
-|-------|---------|-----------|----------|--------|-------|-------|
-| **IQ4_NL (custom)** | **reasoning** | **✅** | **✅ 175 items** | **3** | **0** | **A-** |
-| Q8_0 (custom) | reasoning | ✅ | ❌ await bug | 2 | 0 | B+ |
-| Q8_0 (wiki) | standard | ✅ | ⚠️ 16 items | 5 | 0 | B |
-| IQ4_NL (wiki) | standard | ❌ | — | 10 | 10+ | F |
-| Q8_0 (BF16 KV) | reasoning | ❌ | — | — | 3+ | F |
-**The custom imatrix is the critical factor** — IQ4_NL-custom (A-) vs IQ4_NL-wiki (F) on identical tasks.
 ## Critical: Use F16 KV Cache

 2. **Custom imatrix preserves reasoning weights** — 30% reasoning/self-verification calibration data ensures chain-of-thought and self-correction weights are preserved
 3. **Half the size, better results** — 17GB vs 56GB, fits on a single 24GB GPU
+### How IQ4_NL Was Chosen
+Multiple quantization variants were tested on the same real-world task: building a full-stack AI/ML news aggregator (AI Radar) from a single prompt, using [claw-code](https://github.com/ultraworkers/claw-code) as the agent framework. The project required 4 live API integrations (GitHub, Reddit, HuggingFace, ArXiv), a FastAPI backend, SQLite database, dark-theme SPA frontend, and automated test suite — all at 256k context.
+**Real-world project results (same prompt, same model, different quants):**
+| Variant | Project Completed | Database Persisted Data | Self-Corrected Errors | Loops | Verdict |
+|---------|-------------------|------------------------|----------------------|-------|---------|
+| **IQ4_NL + custom imatrix + F16 KV** | **✅ 7 files** | **✅ 175 items from live APIs** | **3 (all recovered)** | **0** | **Best** |
+| Same IQ4_NL + wiki imatrix + F16 KV | ❌ 0 files | — | 10 (never recovered) | 10+ looping write attempts | Failed |
+| Same model + any quant + BF16 KV | ❌ 0 files | — | — | Looped on basic commands | Failed |
+The IQ4_NL with custom imatrix was the **only variant that produced working database persistence** — all other variants had bugs in async database writes that prevented data from being saved. The custom imatrix preserves the model's self-correction behavior ("wait, that's wrong, let me fix it") that standard calibration destroys at this quantization level.
+**Supporting evidence: 5 automated coding problems (merge intervals, LCS, RPN evaluator, valid parentheses, trapping rain water) tested at both temperature 0.6 and 0.8 — IQ4_NL with custom imatrix scored 5/5 at both temperatures.** BF16 and Q6_K both entered infinite thinking loops on the same problems.
 ## Critical: Use F16 KV Cache