---
base_model: GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER
library_name: gguf
license: apache-2.0
language:
- en
tags:
- gguf
- mtp
- qwen3_5_moe
- conversational
- text-generation
- image-text-to-text
- nsc-ace
- saber
- agentic
- tool-calling
- llama.cpp
- quantized
- base_model:GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER
- base_model:quantized:GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER
pipeline_tag: image-text-to-text
---
[Support this work on Ko-fi](https://ko-fi.com/djlougen)
# Qwen3.6-35B-A3B-NSC-ACE-SABER GGUF MTP
This repository hosts the MTP-oriented llama.cpp/GGUF builds for
`GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER`. The source checkpoint is the full
safetensors model in `GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER`.
These files are rebuilt separately from the first GGUF release using a fresh
MTP-aware conversion path. Use these artifacts when your runtime supports Qwen
MTP / multi-token prediction acceleration.
> **Image/video sidecars:** This repository now includes the restored Qwen3.6 multimodal config, processor/preprocessor files, tokenizer/chat template, safetensors index, and `model-vision-from-qwen3.6-base.safetensors` visual tower sidecar. The existing `.gguf` binaries were not rewritten in this metadata-copy pass.
## Current Status
Files are published only after the rebuilt F16 GGUF verifies as MTP/NextN-capable
from its actual metadata/tensor layout. Qwen HF tensors named `mtp.*` are remapped
by the MTP-aware llama.cpp converter into GGUF `blk.*.nextn.*` tensors plus the
`nextn_predict_layers` metadata key. The upload worker refuses to publish quants
until the verified F16 marker exists.
## Release Snapshot
| Item | Value |
|---|---:|
| Source checkpoint | `GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER` |
| Base model | `Qwen/Qwen3.6-35B-A3B` |
| Format | GGUF for llama.cpp-compatible runtimes |
| Conversion target | MTP-aware GGUF export from the llama.cpp MTP branch |
| Quantization range | F16, Q8_0, Q6_K, Q5_K_M, Q5_K_S, Q4_K_M, Q4_K_S, Q3_K_L, Q3_K_M, Q3_K_S, Q2_K |
| Final source compliance | 98.33% on HarmBench-300 |
| Final source KLD | 0.025383937664711 |
| BFCL average plotted improvement | +2.87 percentage points |
## Benchmark Plots
## BFCL Tool-Calling Check
The source safetensors checkpoint was compared against `Qwen/Qwen3.6-35B-A3B`
on a 40-case BFCL subset: 20 simple and 20 multiple-function prompts. GGUF files
inherit from that checkpoint, but individual quants should be rechecked if exact
tool-call behavior matters.
| Metric | Base | NSC-ACE SABER source |
|---|---:|---:|
| Tool-call rate | 92.50% | 95.00% |
| Function name accuracy | 92.50% | 95.00% |
| Required argument name accuracy | 90.00% | 93.12% |
| Required argument value accuracy | 79.79% | 83.54% |
| Exact required-call accuracy | 75.00% | 77.50% |
## Available Files
| File | Status | Notes |
|---|---|---|
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-F16.gguf` | uploaded | Full GGUF conversion source / highest local fidelity |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q8_0.gguf` | uploaded | Near-full quality, large local file |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q6_K.gguf` | uploaded | High-quality local default if memory allows |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_M.gguf` | uploaded | Strong quality/size balance |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_S.gguf` | uploaded | Smaller Q5 option |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q4_K_M.gguf` | uploaded | Common balanced local target |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q4_K_S.gguf` | uploaded | Smaller Q4 option |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_L.gguf` | uploaded | Lower-memory Q3 option |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_M.gguf` | uploaded | Smaller Q3 balance |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q3_K_S.gguf` | uploaded | Small Q3 option |
| `Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q2_K.gguf` | uploaded | Minimum-size target; quality loss expected |
The uploader refreshes this card as each artifact finishes. Uploaded non-F16
files are deleted from the build pod after upload to stay under the pod volume
quota.
## Which Quant Should I Use?
| Quant | Best fit |
|---|---|
| F16 | Maximum fidelity when disk/RAM are not a concern |
| Q8_0 | Very high fidelity local inference |
| Q6_K | Recommended high-quality local starting point |
| Q5_K_M | Strong balance for quality and size |
| Q4_K_M | Practical default for constrained machines |
| Q3_K_M / Q3_K_S | Low-memory experiments |
| Q2_K | Smallest target; use only when memory is the hard constraint |
For agentic/tool-calling workloads, prefer Q6_K, Q5_K_M, or Q4_K_M when possible.
Very low quants can affect formatting, argument fidelity, and refusal calibration.
## MTP Notes
- These are separate MTP-oriented exports; do not assume the original GGUF repo
exposes MTP behavior in runtimes that require MTP metadata/tensors.
- MTP speedups depend on runtime support. Use a current llama.cpp build.
- Quantized body weights keep `blk.*.nextn.*` tensors at Q8_0 where supported,
because draft-head quality affects speculative acceptance.
- The source model's final release metrics are measured before quantization.
- Quantized files should be re-evaluated if exact compliance/KLD behavior matters.
## Running With llama.cpp
```shell
llama-cli \
-m Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_M.gguf \
-c 32768 \
-ngl 999 \
-p "Write a compact tool plan for indexing a Python repo."
```
For OpenAI-compatible local serving:
```shell
llama-server \
-m Qwen3.6-35B-A3B-NSC-ACE-SABER-MTP-Q5_K_M.gguf \
-c 32768 \
-ngl 999 \
--jinja
```
## Related Repositories
- Full safetensors checkpoint: `GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER`
- Non-MTP GGUF release: `GestaltLabs/Qwen3.6-35B-A3B-NSC-ACE-SABER-GGUF`
- Base model: `Qwen/Qwen3.6-35B-A3B`