README.md

---
license: apache-2.0
model_name: Qwen3.6-27B AEON RYS MaxThinkCoder IQ4_NL GGUF for ik-llama, with experimental MTP artifact
language:
- en
- zh
- multilingual
tags:
- gguf
- qwen3
- qwen3.6
- reasoning
- coding
- academic-writing
- uncensored
- rys
- mtp
- ik-llama
base_model:
- AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored
- Qwen/Qwen3.6-27B
---

# Qwen3.6-27B AEON RYS MaxThinkCoder IQ4_NL GGUF for ik-llama

> Hard runtime requirement: use the custom AEON ik-llama fork:
>
> https://github.com/noonr48/qwen36-aeon-ik-llama
>
> The root GGUF files in this repo are built for that fork. Treat this as an `ik-llama` release, not a stock `llama.cpp` model drop.

> **Newer fine-tunes built on this base — [SignalLatch](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-SignalLatch-GGUF) (a behaviour fine-tune) and [PatchCode](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode-GGUF) (an agentic-coder distil on top of SignalLatch).** This repo is the non-finetuned base.

## Comprehensive Release Record

The full transparent build/test/decision record for this non-finetuned release is here:

https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-15-20/index.html

That page is the detailed companion to this model card. It covers the layer-selection process, quantization-survival checks, runtime profile, practical-test notes, caveats, and the reasoning behind centering this release on the `IQ4_NL` GGUF. This Hugging Face README is the quick download/runtime page; the linked record is where the comprehensive details live.

For future planned RYS releases, this is the expected pattern: a concise model card for use, plus a public companion record for transparency.

SignalLatch behavior fine-tune page:
`https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-signallatch/index.html`

## Project Positioning

The broader goal of this model series is practical local work: capable, functional models that fit realistic hardware and can simply get tasks done. Built on the AEON uncensored base, this release is meant to stay low-friction: less lecturing, less getting in the way, and more focus on completing the task in front of it.

RYS and `IQ4_NL` are the practical part of that goal: preserving useful reasoning and coding behavior in a compact form factor. One 24 GB-class GPU can be enough for a serious local worker. If you have more GPUs, the natural deployment pattern is simple: run more workers in parallel and get more work done at once.

## Runtime + Release Point

Default: use `Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf` for normal inference.

The BF16 artifacts are included for exploration, inspection, conversion, and continued work. The release itself is aimed at practical use through the `IQ4_NL` GGUF: a much smaller file that keeps the model's reasoning and coding behavior close to the BF16 source.

The point of RYS here is not a huge BF16 headline gain. In our testing, the RYS BF16 model only gave roughly a `1%` gain over the Unsloth/official-style BF16 baseline. The useful result was quantization resistance: this specific AEON RYS `15,20` branch, custom runtime path, and `IQ4_NL` quantization held up far better after compression.

We had tried the same broad quantization approach on the official/non-RYS model before, but the outcome was poor: reasoning and coding quality degraded noticeably. The RYS branch was selected because it preserved those behaviors much better under the practical quantized release path.

Experimental: use the MTP GGUF only if you specifically want to test `ik_llama.cpp` multi-token prediction on this model.

Result: the MTP GGUF loads with the expected MTP tensors, but MTP mode was slower than the no-MTP path in our internal graph-split tests, and practical long-output checks favored the normal non-MTP IQ4_NL file.

Runtime: this is an `ik-llama`-targeted release, not a stock `llama.cpp` release.

Required runtime URL: https://github.com/noonr48/qwen36-aeon-ik-llama

Download/build that fork first; the recommended GGUF is intended for this runtime path.

RYS weight representation note: this release uses materialized RYS. The copied `15,20` window is exported as normal GGUF layer tensors, so output layers `20..24` have their own tensors copied from source layers `15..19` instead of aliasing those source buffers at runtime. That is intentional for this release: it keeps the checkpoint, GGUF conversion, quantization path, and downstream modification/fine-tune workflows explicit. A procedural/aliased runtime could be explored in future work to reduce duplicate weight memory, but the current `IQ4_NL` artifact has already been tested as stable and reliable, so I do not plan to change its representation without rebuilding and re-verifying the release.

### Recommended llama-server profile

Use the custom AEON ik-llama fork and start with this public deployment shape for the recommended non-MTP `IQ4_NL` GGUF:

```bash
./build/bin/llama-server \
  -m /path/to/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf \
  -c 65536 \
  -ngl 999 \
  -np 1 \
  -fa on \
  -sm graph \
  --temp 0.7 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 0
```

For long-context use, the same profile can be run with a larger context. The default KV type is `f16`; we have also tested the `f16` KV path up to about `160k` context without seeing the earlier failure pattern:

```bash
  -c 131072
```

Practical single-GPU deployment note: the Q4_NL release is small enough for practical use on a single RTX 3090 / 24 GB-class card. In an observed reference profile, roughly `160k` context with default/FP16 KV fit at about `20.3 GiB` total VRAM. Treat this as a deployment reference point, not a guaranteed cross-hardware memory benchmark.

For the later 131k practical canvas-agent comparison, we used FP32 KV as a conservative isolation setting to make sure any failure would not be caused by KV precision:

```bash
  -c 131072 \
  -ctk f32 \
  -ctv f32
```

Runtime notes:
- `-c 65536` is the practical baseline profile; `-c 131072` is the long-context comparison profile.
- FP32 KV was used for the 131k canvas comparison as a conservative validation setting, not because FP16 KV is known-bad. For normal practical use, start with default/FP16 KV and reduce context or KV precision only if your system needs it.
- `-sm graph` is the tested strength-sweep / comparison path. If you are running a RAM-cache parallel deployment, use the separate setup below.
- Do not pass `--ctx-checkpoints 0` for RAM prompt-cache deployments. The prompt may still match, but without recurrent context checkpoints the server can fall back to full prompt reprocessing.
- Use Jinja plus DeepSeek reasoning formatting for the OpenAI-compatible chat path.

### Parallel RAM-cache and agent-harness setup

The custom AEON ik-llama fork now includes a runtime adjustment for explicit slot lanes and RAM prompt-cache reuse. This matters for agent harnesses such as OpenCode, where you may want one long-running worker on `slot0` and another on `slot1` instead of letting requests bounce between slots.

Generic two-slot server shape:

```bash
./build/bin/llama-server \
  -m /path/to/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf \
  -c 360000 \
  -np 2 \
  -ngl 999 \
  -b 256 \
  -ub 64 \
  -fa on \
  -sm layer \
  -ctk f16 \
  -ctv f16 \
  --temp 0.7 \
  --jinja \
  --reasoning-format deepseek \
  --reasoning-budget 0 \
  --alias qwen36-aeon-rys-q4nl \
  -cram 65536
```

How to read that setup:
- `-np 2` creates two parallel slots.
- `-c 360000` is the total server context budget; with two slots, each lane receives roughly half.
- `-cram 65536` enables RAM prompt cache. The value is MiB and is server-wide, so adjust it to your host memory budget.
- keep context checkpoints enabled. Do not add `--ctx-checkpoints 0` if you want RAM prompt-cache restores to avoid full prompt reprocessing.
- use `-sm layer` for this two-GPU / two-slot RAM-cache path. On this recurrent/hybrid path, `-sm graph` disables the checkpoints needed for actual restore.
- set a stable `--alias`, then point the agent harness at `<alias>-slot0` and `<alias>-slot1`.

Minimal OpenCode-style provider shape:

```json
{
  "provider": {
    "aeon-rys-q4nl-p2": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "AEON RYS Q4_NL p2",
      "options": {
        "baseURL": "http://<host>:<port>/v1",
        "timeout": false
      },
      "models": {
        "qwen36-aeon-rys-q4nl-slot0": {
          "name": "AEON RYS Q4_NL slot0",
          "reasoning": true,
          "interleaved": { "field": "reasoning_content" },
          "limit": { "context": 180096, "input": 171904, "output": 8192 }
        },
        "qwen36-aeon-rys-q4nl-slot1": {
          "name": "AEON RYS Q4_NL slot1",
          "reasoning": true,
          "interleaved": { "field": "reasoning_content" },
          "limit": { "context": 180096, "input": 171904, "output": 8192 }
        }
      }
    }
  }
}
```

The important part is the model-name suffix. If your server alias is `qwen36-aeon-rys-q4nl`, the OpenAI-compatible request model names `qwen36-aeon-rys-q4nl-slot0` and `qwen36-aeon-rys-q4nl-slot1` pin requests to the matching server slot.

Rationale: a long-running agent benefits from a stable context lane. If a harness sends related work sometimes to slot 0 and sometimes to slot 1, the live KV/recurrent checkpoint state for that lane may not match the next request and the server may need to reprocess more of the prompt. Slot suffixes make the lane explicit. In the unpatched pinned-slot path we tested, those suffixes selected the slot but skipped RAM prompt-cache save/load. The current fork fixes that by running the RAM prompt-cache path for pinned slots too.

Verification we performed: direct A/B/A requests against the current fork showed the expected cache behavior. The repeated `A` request produced `found better prompt`, `prompt cache load took ...`, and `restored context checkpoint` in the server log instead of a full prompt reprocess. When `--ctx-checkpoints 0` was present, the prompt match could still be found, but the server had no checkpoint state to restore and fell back to reprocessing. That is why the RAM-cache profile above leaves checkpoints enabled.

Full runtime and release details are in the companion record:

https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-15-20/index.html

Behavior fine-tune note: a newer minimal behavioral fine-tune of this RYS line is available as `SignalLatch`:

https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-SignalLatch-GGUF

It keeps the same practical `IQ4_NL` deployment idea, but adds a small checkpoint-386 / strength-0.10 behavior merge trained around a `Review -> Align -> Latch -> Repair -> Confirm` loop for coding agents: scoped context review, goal and constraint alignment, waiting for concrete tool/command signals, targeted repair from evidence, and focused validation.

### Which File Should I Download?

- **Normal inference / recommended default:**
  [`Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/blob/main/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf)
  Runtime: [`qwen36-aeon-ik-llama`](https://github.com/noonr48/qwen36-aeon-ik-llama)
- **MTP runtime testing only:**
  [`Qwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/blob/main/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf)
  Runtime: [`qwen36-aeon-ik-llama`](https://github.com/noonr48/qwen36-aeon-ik-llama) with Qwen3.5/Qwen3.6 MTP support
- **BF16 GGUF exploration / conversion checks:**
  [`Qwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.gguf`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/blob/main/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.gguf)
- **Transformers, LoRA, SFT, continued training, or inspection:**
  [`bf16-safetensors/`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/tree/main/bf16-safetensors)

### Quality / Compression

The released IQ4_NL GGUF is about `70%` smaller on disk than the BF16 GGUF:
`54G` BF16 -> `16G` IQ4_NL.

This is a mixed validation snapshot, not a broad leaderboard benchmark. It compares the RYS BF16 source artifact against the released RYS `IQ4_NL` file.

The important read is the compression behavior: after the drop from BF16 to `IQ4_NL`, the mixed four-probe score changed by less than `1%` relative.

| probe | RYS BF16 | released RYS IQ4_NL |
|---|---:|---:|
| mixed 4-probe mean | `0.7299` | `0.7244` |
| `math_16` | `0.8421` | `0.7897` |
| `eq_16` | `0.7123` | `0.7111` |
| `math_4` | `0.4851` | `0.5170` |
| `gsm8k_5` | `0.8800` | `0.8800` |

Mixed four-probe snapshot change: `-0.0055` absolute, about `-0.75%` relative.

That is the intended claim for this release: RYS made this practical Q4_NL deployment path much more resistant to reasoning/coding degradation. The BF16 RYS model is available for exploration, but the main artifact is the compressed `IQ4_NL` GGUF.

Practical coding-agent test: we also used this non-finetuned RYS `IQ4_NL` model in an isolated Krita-like canvas-agent task: a local app with layers, brushes, transforms, opacity controls, and a local-only AI image generation stub. Under temp `0.7`, `131072` context, conservative FP32 KV cache, flash-attention, graph-split setup, the first attempt hit an invalid tool/diff formatting failure before producing usable root files, but a clean retry completed with `rc=0` and verifier score `1.0`. FP32 KV was used here to isolate the model/task behavior from KV precision questions; separate long-context checks with FP16/default KV have reached about `160k` context without showing the earlier failure pattern. This is a useful practical result: the compressed RYS path can complete the coding-agent task, while the later SignalLatch behavior fine-tune was added to improve reliability and completion discipline on top of this base.

### Speed / MTP

These are context-specific internal snapshots, not a normalized cross-hardware benchmark or a public speed claim. Hardware identifiers and local machine details are intentionally omitted here; treat the rows as relative notes for the tested runtime shapes only.

The first two rows compare the custom runtime path against an internal, non-released patched upstream-style comparison path. The MTP rows are a separate graph-split runtime experiment and should not be read as a direct hardware comparison.

| check | setup | decode speed |
|---|---|---:|
| recommended custom `ik-llama` deployment | graph split, total server `ctx=409600`, `np=2` (~204800 token KV budget/slot), `f32/f32` KV | `39.37 tok/s` |
| internal non-released patched upstream-style comparison path | layer-style comparison, `ctx=4096`, `np=1`, `f16` KV | `22.51 tok/s` |
| no-MTP short generation | graph split, 192-token check | `48.68 tok/s` |
| naive MTP draft-1 short generation | graph split, same 192-token check | `38.16 tok/s` |
| adaptive MTP short generation | graph split, same 192-token check | `45.36 tok/s` |
| adaptive MTP longer generation | graph split, 768-token check | `46.95 tok/s` vs `48.71 tok/s` no-MTP |

Note: the `SpeedBoosted` filename is legacy packaging wording for the custom `ik_llama` release context. Treat this file as experimental MTP, not as a speedup claim.

### Model Details

- Public codename: `MaxThinkCoder`
- Source model: [`AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored`](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored)
- RYS branch: AEON-derived `blocks:15,20`
- Focus: coding, technical reasoning, academic-style writing, and long-context English work
- Required runtime: custom [`qwen36-aeon-ik-llama`](https://github.com/noonr48/qwen36-aeon-ik-llama)
- RYS build guide: [`docs/rys-layer-duplication-guide`](https://github.com/noonr48/qwen36-aeon-ik-llama/tree/main/docs/rys-layer-duplication-guide)
- Not a generic safe-default chat model
- Not a stock `llama.cpp` release

Tool-calling note: use a build of the linked `ik-llama` fork that includes the May 2026 duplicate `tool_call` filtering. It deduplicates repeated identical `tool_calls` server-side.


## Experimental MTP Details

An optional MTP-capable IQ4_NL GGUF is included for testing ik_llama.cpp multi-token prediction:

[`Qwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/blob/main/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf)

Use it only with an `ik_llama.cpp` build that has Qwen3.5/Qwen3.6 MTP support. The tested MTP path used:

```bash
-sm graph -mtp --draft-max 1 --draft-p-min 0.0 --draft-min 0 -b 128 -ub 32 -ctk f16 -ctv f16 -fa on -gr
```

What our tests found:
- MTP is structurally intact in this artifact: it has `qwen35.nextn_predict_layers = 1` and the `blk.69.nextn.*` tensors.
- This uploaded MTP file is the MTP-aware imatrix requant, so the MTP tail block is covered by the imatrix instead of falling back to missing-imatrix behavior for the `blk.69` matrix tensors.
- We tried graph split, MTP graph reuse, avoiding unnecessary logits in MTP warmup/update paths, safer KV/recurrent restore handling, GPU-side draft argmax, and a runtime adaptive MTP gate.
- In our tested graph-split configurations, MTP remained below the non-MTP path: `38.16 tok/s` naive MTP and `45.36 tok/s` adaptive MTP vs `48.68 tok/s` non-MTP on the 192-token check.
- A 768-token check showed the same pattern: `46.95 tok/s` adaptive MTP vs `48.71 tok/s` non-MTP.
- GPU-side argmax was neutral to slightly slower in this setup.
- Practical quality testing also favored the original non-MTP release: fewer long-output repeat penalties and better default behavior.

What this means in practice:
- the MTP artifact loads and can be useful for runtime/MTP research
- the linked `ik-llama` fork has Qwen3.5/Qwen3.6 MTP support and still supports the non-MTP GGUF
- the recommended model for actual use is still the normal non-MTP `IQ4_NL-ik-llama-custom-mixed` GGUF
- MTP may become useful later if the MTP head is calibrated or fine-tuned for the RYS/AEON hidden-state distribution

Recommendation: download the MTP file if you specifically want to test MTP. For normal use, use the original non-MTP `IQ4_NL-ik-llama-custom-mixed` GGUF.


## BF16 GGUF And Safetensors

A BF16 GGUF is also included for people who want to explore the unquantized GGUF-side artifact from the same released `15,20` RYS branch:

[`Qwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.gguf`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/blob/main/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.gguf)

Use this if you want a GGUF reference build, local conversion/testing, or to compare quantization behavior against the released `IQ4_NL` GGUF. It is here for exploration and downstream work. For normal inference, the `IQ4_NL` file is the practical target. For Transformers/LoRA/SFT workflows, use the [`bf16-safetensors/`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/tree/main/bf16-safetensors) folder instead.

Size note:
- BF16 GGUF: about `54G`
- IQ4_NL GGUF: about `16G`

The quality/compression snapshot is shown at the top of this card. The short version is that the released IQ4_NL file is the practical inference target, while the BF16 artifacts are included for reference, training workflows, and conversion work.

The original HF-format BF16 checkpoint for the released `15,20` RYS branch is included here:

[`bf16-safetensors/`](https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF/tree/main/bf16-safetensors)

Use the files in that folder for Transformers-based work such as LoRA, SFT, continued training, or conversion into another training format. Use the GGUF file in the repo root for `ik-llama` inference.

Loading example:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF"
subfolder = "bf16-safetensors"

tok = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder=subfolder,
    torch_dtype="bfloat16",
    device_map="auto",
    trust_remote_code=True,
)
```

RYS note for anyone rebuilding or modifying this checkpoint: the config is part of the model. The BF16 folder keeps the corrected hybrid-stack metadata for the `15,20` insert, including `text_config.num_hidden_layers = 69` and a 69-entry `text_config.layer_types` list. Do not change the layer count without remapping `layer_types` to the same layer order as the tensors.

## Runtime Compatibility

We did build and benchmark an internal standard-typed comparison artifact.
We are not releasing it as a public `llama.cpp` file.

Why:
- the main model this project is about is the custom mixed GGUF, which needs the forked `ik-llama` runtime
- even the internal standard-typed comparison path was only validated on a patched upstream-style `llama.cpp`, not clean stock mainline
- to avoid implying stock `llama.cpp` support, this release does not include that internal comparison artifact

So the intended reading is simple:
- this repo releases the `ik-llama`-targeted model
- if you want plain stock `llama.cpp`, this is not that release

## Project Focus And RYS Choice

This is a deliberately narrow project. The target is a practical Q4-class English-first model optimized for coding, technical reasoning, academic-style writing, and long-context work.

This release came from the AEON-derived `15,20` RYS branch. That branch was the practical release target because it quantized cleanly and held up as the best balanced candidate for this experiment.

The honest summary is that RYS was not a dramatic BF16-only upgrade in this run. The BF16-side gain over the Unsloth/official-style BF16 baseline was only around `1%`. The practical value was that the RYS branch made this model much more resistant to quality loss under the `IQ4_NL` quantization path than the non-RYS/official-style model had been.

The source path is uncensored, so this is not a generic safe-default chat model.

## Imatrix calibration profile

The quantization was deliberately biased toward reasoning and technical work.

Heuristic calibration breakdown:
- `math_reasoning`: `5,688` chunks, `1,706,070` chars (`36.0%`)
- `code_technical`: `3,518` chunks, `1,343,392` chars (`28.4%`)
- `experiment_docs`: `808` chunks, `224,169` chars (`4.7%`)
- `writing_chat`: `387` chunks, `164,097` chars (`3.5%`)
- `other`: `5,139` chunks, `1,249,396` chars (`26.4%`)

Practical read:
- heavy focus on reasoning math, code, technical prose, and experiment artifacts
- very little emphasis on generic social chat

## Use case

Recommended:
- coding
- technical reasoning
- academic-style writing
- long-context English work

Not recommended as a generic safe-default chat model.
This branch came from an uncensored source path.