GGUF
English
Chinese
multilingual
qwen3
qwen3.6
reasoning
coding
coding-agent
academic-writing
uncensored
rys
lora
iq4_nl
bf16
conversational
Instructions to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: ./llama-cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Use Docker
docker model run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- LM Studio
- Jan
- Ollama
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Ollama:
ollama run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- Unsloth Desktop
- Pi
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Docker Model Runner:
docker model run hf.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
- Lemonade
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Run and chat with the model
lemonade run user.Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF-BF16
List all available models
lemonade list
- Hermes Agent
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF:BF16" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -46,7 +46,7 @@ Upgrade target:
|
|
| 46 |
|
| 47 |
SignalLatch was already close to its BF16 source on the mixed probe snapshot. PatchCode keeps that small-form-factor Q4_NL path as the main deployment target and tests whether the agentic-coder distil improves practical coding-agent behaviour on top of it.
|
| 48 |
|
| 49 |
-
Practical eval: under a hardened 5-seed, same-condition bake-off (160k-token real-world multi-file build as the discriminator — single-shot coding gates saturate and were rejected), PatchCode `IQ4_NL` tied `BF16` within noise on build, long-context,
|
| 50 |
|
| 51 |
Release files:
|
| 52 |
- `Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf`
|
|
@@ -159,27 +159,26 @@ Runtime notes:
|
|
| 159 |
|
| 160 |
These numbers are from an internal practical coding-agent build matrix. This is not an academic benchmark. The discriminator is a 160k-token real-world multi-file build, scored multi-seed (single-shot coding gates saturate on this model family and were rejected).
|
| 161 |
|
| 162 |
-
| candidate | build (KritaLite) | long-context | discipline (style) |
|
| 163 |
-
|---|---:|---:|---:|---
|
| 164 |
-
| **PatchCode IQ4_NL (reasoning imatrix)** | `0.920` (±0.067) | `0.975` | `0.842` (±0.333) |
|
| 165 |
-
| BF16 (control) | `0.867` | `0.942` | `0.931` |
|
| 166 |
-
| Q8_0 | `0.867` | `0.969` | `0.742` |
|
| 167 |
-
| mixed-recipe (promoted attn) | `0.907–0.933` | `0.935` | `0.69–0.90` |
|
| 168 |
|
| 169 |
Read:
|
| 170 |
-
- PatchCode `IQ4_NL` ties `BF16` within noise on build, long-context,
|
| 171 |
- Near-lossless `Q8_0` and promoted-attention mixed recipes bought **no measurable edge** and cost 2–6× the size.
|
| 172 |
- This is a tentative long-term default, not a claim that the fine-tune is solved. Build has ±0.067–0.13 run-to-run variance and discipline ±0.3 on this suite; a 3-seed single-condition run shipped a *false* winner (a mixed recipe scored 0.933 once, never reproduced). Only 5+ seed same-condition head-to-heads + non-noise axes (size, recipe safety) reliably tiebreak. `IQ4_NL` wins on size + the plain-quant recipe (no promotion risk).
|
| 173 |
|
| 174 |
-
|
| 175 |
|
| 176 |
-
|
|
| 177 |
-
|---|---:|---:|---:|
|
| 178 |
-
|
|
| 179 |
-
|
|
| 180 |
-
| mixed (c76) | 100% (40/40) | 6.6 | 0.4 | 0% | 0% |
|
| 181 |
|
| 182 |
-
|
| 183 |
|
| 184 |
## IQ4_NL vs BF16 — head-to-head (5-seed, same-condition)
|
| 185 |
|
|
@@ -190,7 +189,6 @@ The released `IQ4_NL` is what most people should run. It was confirmed against t
|
|
| 190 |
| build (KritaLite, 5-seed) | `0.920` (±0.067) | `0.867` | IQ4_NL nominally higher; gap `0.053` ≈ within the ±0.067 noise floor |
|
| 191 |
| long-context (4 tasks) | `0.975` | `0.942` | both near ceiling; IQ4_NL marginally higher |
|
| 192 |
| discipline (action-first style, 5-seed) | `0.842` (±0.333) | `0.931` | BF16 nominally higher; gap `0.089` ≪ ±0.333 spread → within noise |
|
| 193 |
-
| autonomous-loop convergence | `100%` (40/40), 0% halluc/stall | — | preserved |
|
| 194 |
| **size on disk** | **`16.6 G`** | `57.6 G` | **IQ4_NL is ~71% smaller** |
|
| 195 |
|
| 196 |
Read:
|
|
@@ -207,11 +205,10 @@ A near-lossless `Q8_0` was built and tested head-to-head (5-seed, same-condition
|
|
| 207 |
| build (KritaLite) | `0.867` (±0.133) | `0.920` (±0.067) | — |
|
| 208 |
| long-context | `0.969` | `0.975` | — |
|
| 209 |
| discipline (style) | `0.742` (±0.292) | `0.842` (±0.333) | — |
|
| 210 |
-
| autonomous-loop convergence | `100%` (40/40) | `100%` (40/40) | — |
|
| 211 |
| **size on disk** | `29 G` | `16.6 G` | Q8 is ~1.7× larger |
|
| 212 |
|
| 213 |
Read:
|
| 214 |
-
- Q8_0 is **at or below** the released IQ4_NL on build, long-context, and discipline (all within noise)
|
| 215 |
- Near-lossless precision buys nothing measurable on this suite (consistent with the finding that build here is **not** precision-limited — it's ceiling-limited and noisy). Releasing a 29 G file that ties a 16.6 G file would be wasted bytes.
|
| 216 |
- Attention-promotion mixed recipes were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.
|
| 217 |
|
|
|
|
| 46 |
|
| 47 |
SignalLatch was already close to its BF16 source on the mixed probe snapshot. PatchCode keeps that small-form-factor Q4_NL path as the main deployment target and tests whether the agentic-coder distil improves practical coding-agent behaviour on top of it.
|
| 48 |
|
| 49 |
+
Practical eval: under a hardened 5-seed, same-condition bake-off (160k-token real-world multi-file build as the discriminator — single-shot coding gates saturate and were rejected), PatchCode `IQ4_NL` tied `BF16` within noise on build, long-context, and discipline, at ~⅓ the size. See the eval snapshot below.
|
| 50 |
|
| 51 |
Release files:
|
| 52 |
- `Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf`
|
|
|
|
| 159 |
|
| 160 |
These numbers are from an internal practical coding-agent build matrix. This is not an academic benchmark. The discriminator is a 160k-token real-world multi-file build, scored multi-seed (single-shot coding gates saturate on this model family and were rejected).
|
| 161 |
|
| 162 |
+
| candidate | build (KritaLite) | long-context | discipline (style) | size |
|
| 163 |
+
|---|---:|---:|---:|---:|
|
| 164 |
+
| **PatchCode IQ4_NL (reasoning imatrix)** | `0.920` (±0.067) | `0.975` | `0.842` (±0.333) | `16.6 G` |
|
| 165 |
+
| BF16 (control) | `0.867` | `0.942` | `0.931` | `57.6 G` |
|
| 166 |
+
| Q8_0 | `0.867` | `0.969` | `0.742` | `29 G` |
|
| 167 |
+
| mixed-recipe (promoted attn) | `0.907–0.933` | `0.935` | `0.69–0.90` | `20–24 G` |
|
| 168 |
|
| 169 |
Read:
|
| 170 |
+
- PatchCode `IQ4_NL` ties `BF16` within noise on build, long-context, and discipline (5-seed, same-condition).
|
| 171 |
- Near-lossless `Q8_0` and promoted-attention mixed recipes bought **no measurable edge** and cost 2–6× the size.
|
| 172 |
- This is a tentative long-term default, not a claim that the fine-tune is solved. Build has ±0.067–0.13 run-to-run variance and discipline ±0.3 on this suite; a 3-seed single-condition run shipped a *false* winner (a mixed recipe scored 0.933 once, never reproduced). Only 5+ seed same-condition head-to-heads + non-noise axes (size, recipe safety) reliably tiebreak. `IQ4_NL` wins on size + the plain-quant recipe (no promotion risk).
|
| 173 |
|
| 174 |
+
PatchCode vs the base it was distilled from (15-case behaviour rubric; adapter-off as the "strength 0" anchor = the SignalLatch base):
|
| 175 |
|
| 176 |
+
| variant | rubric score | avg output tokens | avg time/case |
|
| 177 |
+
|---|---:|---:|---:|
|
| 178 |
+
| base (SignalLatch) | `0.486` | `311` | `34s` |
|
| 179 |
+
| PatchCode (λ=0.5) | `0.617` | `91` | `13s` |
|
|
|
|
| 180 |
|
| 181 |
+
PatchCode scores higher while emitting ~⅓ the tokens — the base rambled (~311 tokens of hedging preamble), PatchCode was terse and on-target. (Behaviour rubric, not a multi-turn agent turn-count; λ=0.5 is the sweet spot — higher strengths also got terse but fell *below* the base.)
|
| 182 |
|
| 183 |
## IQ4_NL vs BF16 — head-to-head (5-seed, same-condition)
|
| 184 |
|
|
|
|
| 189 |
| build (KritaLite, 5-seed) | `0.920` (±0.067) | `0.867` | IQ4_NL nominally higher; gap `0.053` ≈ within the ±0.067 noise floor |
|
| 190 |
| long-context (4 tasks) | `0.975` | `0.942` | both near ceiling; IQ4_NL marginally higher |
|
| 191 |
| discipline (action-first style, 5-seed) | `0.842` (±0.333) | `0.931` | BF16 nominally higher; gap `0.089` ≪ ±0.333 spread → within noise |
|
|
|
|
| 192 |
| **size on disk** | **`16.6 G`** | `57.6 G` | **IQ4_NL is ~71% smaller** |
|
| 193 |
|
| 194 |
Read:
|
|
|
|
| 205 |
| build (KritaLite) | `0.867` (±0.133) | `0.920` (±0.067) | — |
|
| 206 |
| long-context | `0.969` | `0.975` | — |
|
| 207 |
| discipline (style) | `0.742` (±0.292) | `0.842` (±0.333) | — |
|
|
|
|
| 208 |
| **size on disk** | `29 G` | `16.6 G` | Q8 is ~1.7× larger |
|
| 209 |
|
| 210 |
Read:
|
| 211 |
+
- Q8_0 is **at or below** the released IQ4_NL on build, long-context, and discipline (all within noise) — and it costs ~75% more disk.
|
| 212 |
- Near-lossless precision buys nothing measurable on this suite (consistent with the finding that build here is **not** precision-limited — it's ceiling-limited and noisy). Releasing a 29 G file that ties a 16.6 G file would be wasted bytes.
|
| 213 |
- Attention-promotion mixed recipes were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.
|
| 214 |
|