Squashed: Release 2026-08-15:2

.gitattributes

@@ -0,0 +1,3 @@
+*.gguf filter=lfs diff=lfs merge=lfs -text
+*.md text -whitespace
+/LICENSE text eol=lf -whitespace

.gitignore

@@ -0,0 +1,3 @@
+.*
+!.git?*
+*~

BF16/Qwen3.8-2.4T-A95B-MTP-ONLY-BF16-00001-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37660e1d16a3192d750952f74fb98885feed045c4748ca2d9e4ea18cb3cbe343
+size 38439156192

BF16/Qwen3.8-2.4T-A95B-MTP-ONLY-BF16-00002-of-00002.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afeb3a9cc6891d78fdd16152c67d9e5f458d3e6e9f317aac1df92a266b004351
+size 22473313600

LICENSE

@@ -0,0 +1,16 @@
+Qwen3.8-Max License
+
+Copyright (c) 2026 Qwen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software, including the model weights, parameters, configuration files, inference code and associated documentation files (collectively, the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, sell, deploy, host, fine-tune, and create derivative works from (collectively, "Use" or "Using") copies of the Software; and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. If the Software (or any derivative works thereof) is Used for any of the licensee's commercial products or services that have more than 100,000,000 monthly active users or US$ 20,000,000 (or equivalent in other currencies) monthly revenue, respective model name must be prominently displayed on the user interface of such product or service; and,
+
+2. If the licensee or any of its affiliates conducts a Model as a Service or AI Work Assistant business, and the aggregate revenue of the licensee and its affiliates exceeds US$50,000,000 (or the equivalent amount in any other currencies) during any consecutive twelve (12) months, the licensee shall obtain a separate license from Qwen before Using the Software or its derivative works for any commercial purpose. The foregoing requirement shall not apply to the licensee's internal Use of the Software, provided that such Use does not make the Software, its outputs, or its underlying model capabilities available to any third party.
+
+"Model as a Service" means giving a third party access to language model inference or fine-tuning (e.g., via API or a hosted endpoint) in a manner that allows such third parties to exercise meaningful control over the inputs, parameters, or training data. This does not include the mere relaying of requests to models hosted by other third parties. 
+“AI Work Assistant” means an independent AI-powered product primarily designed for AI-assisted coding or office productivity (e.g., Qoder and QwenWork). It does not include: (a) a single-purpose AI tool (such as an AI translation tool); (b) an AI assistant primarily designed for a domain other than coding or office productivity (such as Taobao AI Shopping Assistant or AMap AI Chat); or (c) an AI assistant that is a feature of a product whose primary purpose is not AI-assisted coding or office productivity.
+
+THE SOFTWARE AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL QWEN, ITS AFFILIATES OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. THE USE OF THE SOFTWARE MUST COMPLY WITH APPLICABLE LAWS AND REGULATIONS, AND MUST NOT INFRINGE THE INTELLECTUAL PROPERTY RIGHTS OF ANY THIRD PARTY.
+
+For any questions regarding this license, please contact model-business@notice.qwencloud.com.

Qwen3.8-2.4T-A95B-MTP-ONLY-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13ab30233ec48a61ea18cc809742f67c42db65c816be934e84b2a008b70a86ce
+size 19897255520

Qwen3.8-2.4T-A95B-MTP-ONLY-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01550a1504d7edf7d3686a5f852114c211075834a83c2cd895ec55d64717a30b
+size 22371370592

Qwen3.8-2.4T-A95B-MTP-ONLY-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a7cd74c135c238025390c335a15e5b19534d5198cffbe3b81a68fbe4213f28b
+size 25000117856

Qwen3.8-2.4T-A95B-MTP-ONLY-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d6cf03a5e28d46c867a03d5df5c9ffacc4a861d459a15465bfedeaca2a759b8
+size 32372852320

README.md

@@ -0,0 +1,166 @@
+---
+library_name: gguf
+license: other
+license_name: qwen3.8-max
+license_link: https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/207bd685a7e3696cfaff12ded7c6a7ea0f88c996/LICENSE
+base_model:
+- Qwen/Qwen3.8-2.4T-A95B
+base_model_relation: quantized
+tags:
+- qwen3.5
+- qwen3.8
+- speculative-decoding
+- mtp
+- mtp only
+- gguf
+---
+
+MTP-only GGUF subset of Qwen3.8 models
+=======================================
+
+This is a supplement for Qwen3.8-2.4T-A95B-based models (including fine-tunes and
+abliterated models) **without MTP tensors**.
+
+This repository contains an **MTP-only subset** of [Qwen/Qwen3.8-2.4T-A95B](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B)
+which provides the draft model for speculative decoding in the GGUF format.
+
+It accelerates token generation using speculative decoding with the draft model
+from the original Qwen model.  In most cases, this is sufficient to accelerate
+Qwen-based derivative models even if this draft model is not trained from them.
+
+Note that however, the performance metrics heavily depend on the derivative
+model you use, your machine and your MTP settings.
+
+**Benchmark it before blindly trusting it.**
+
+
+Using this Model
+-----------------
+
+It can be used in two ways:
+
+1.  As a separate draft model file (Method 1)
+2.  As a donor for grafting the draft model into a Qwen-based model
+    (Method 2; Recommended)
+
+### Method 1: Separate Draft Model File
+
+It is easy to begin with but memory-inefficient as it does not share
+some tensors with the original model.
+
+If you find the draft model can accelerate a Qwen-based model you use, grafting
+the draft model (Method 2) is recommended (note: switching to Method 2 may
+slightly change the acceptance rate).
+
+If you use `llama-server`, you may configure like this:
+
+```
+llama-server \
+    --model       Qwen3.6-27B-finetune-Q4_K_M.gguf \
+    --model-draft Qwen3.6-27B-MTP-ONLY-Q4_K_M.gguf \
+    ... \
+    --spec-type draft-mtp
+    --spec-draft-n-max 4
+```
+
+`--model` specifies the original Qwen-based model and
+*new* `--model-draft` specifies a file from this repository.
+
+You also need `--spec-type draft-mtp` to enable the draft model.
+
+Once the draft model is enabled, you may configure the rest of MTP options
+as you like (in this example, custom `--spec-draft-n-max` is specified).
+
+### Method 2: Grafting the Draft Model (Using this as a Donor)
+
+This is recommended.
+
+First, download [`convert.py`](https://gist.github.com/buzz/1c439684d5e3f36492ae9f64ef7e3f67/7e1d6f929e141f4977f89f99e6a7df39f41eeffa)
+written by [@buzz](https://github.com/buzz) to transplant MTP tensors.
+You may also need to install some dependencies required by this script.
+
+Then, you can run this script like:
+
+```sh
+# ./convert.py INPUT MTP OUTPUT
+python3 convert.py \
+    Qwen3.6-27B-finetune-Q4_K_M.gguf \
+    Qwen3.6-27B-MTP-ONLY-Q4_K_M.gguf \
+    Qwen3.6-27B-finetune-Q4_K_M+MTP.gguf
+```
+
+The second argument of `convert.py` is a GGUF file (donor) from this repository.
+
+Once the grafted GGUF file is created, you may use this like a regular
+Qwen model with embedded draft model.
+
+This is an example for `llama-server` users.
+
+```
+llama-server \
+    --model Qwen3.6-27B-finetune-Q4_K_M+MTP.gguf \
+    ... \
+    --spec-type draft-mtp
+    --spec-draft-n-max 4
+```
+
+The output of `convert.py` must be specified as the model file name
+(**DO NOT** use `--model-draft` in this case).
+
+`--spec-type draft-mtp` enables the draft model transplanted into the main one.
+
+
+Additional Quantization (`Q4_K_M`, `Q5_K_M`, `Q6_K` and `Q8_0`)
+----------------------------------------------------------------
+
+Quantized GGUF files are provided so that deploying the draft model easier.
+
+*It is not required to match the quantization level.*
+For instance, you may pair `Q6_K`-quantized draft model with
+the `Q4_K_S`-quantized main model.
+
+
+Conversion Process
+-------------------
+
+*   Tools: [llama.cpp](https://github.com/ggml-org/llama.cpp) ([b10380](https://github.com/ggml-org/llama.cpp/releases/tag/b10380))
+*   With: Patched `conversion/base.py`  
+    The `if` block right after `# verify tensor name presence and identify potentially missing files` is commented out.
+
+This modification is performed because the author of this repository downloaded
+only a subset of the full Qwen model while the original `convert_hf_to_gguf.py`
+expects the full model.
+
+The `--mtp` option of `convert_hf_to_gguf.py` is the crucial part of this
+conversion process because this option does exactly what the author expects:
+create an MTP-only GGUF subset.
+
+For additional quantization, the `llama-quantize` tool (llama.cpp) is used as-is.
+
+
+License and Copyright
+----------------------
+
+For all GGUF files under this repository,
+[the license terms of the original Qwen model (Qwen3.8-Max License)](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/207bd685a7e3696cfaff12ded7c6a7ea0f88c996/LICENSE)
+applies (as the author of this repository did not perform any changes
+significant enough for own copyright):
+
+> Copyright (c) 2026 Qwen
+
+This README file is licensed under the terms of [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/).
+
+> Copyright 2026 a4lg.
+
+
+Links: Sources
+---------------
+
+*   The Original Model: [Qwen/Qwen3.8-2.4T-A95B](https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/tree/207bd685a7e3696cfaff12ded7c6a7ea0f88c996)
+
+
+Links: All MTP Subsets: Qwen3.8 (incl. non-open-source one)
+------------------------------------------------------------
+
+*   [Qwen/Qwen3.8-27B](https://huggingface.co/a4lg/Qwen3.8-27B-MTP-ONLY-GGUF)
+*   [Qwen/Qwen3.8-2.4T-A95B](https://huggingface.co/a4lg/Qwen3.8-2.4T-A95B-MTP-ONLY-GGUF)