--- license: apache-2.0 library_name: llama.cpp pipeline_tag: text-generation base_model: - prism-ml/Ternary-Bonsai-27B-gguf tags: - gguf - llama.cpp - qwen35 - ternary - tq2_0 - mtp - nextn - speculative-decoding - local-llm - self-contained - q2_k --- # Ternary Bonsai 27B with Embedded MTP — GGUF > **One target, one embedded NextN predictor, one deployment artifact per representation.** `qwen35` · 64 target blocks · 1 embedded NextN block · 866 tensors · Apache-2.0 --- ## Release summary This repository publishes two self-contained GGUF representations of the same logical model graph: 1. **`TQ2_0` — primary, native ternary representation** 2. **`Q2_K` — compatibility-oriented standard representation** Each file contains both the 27B-class target and its embedded Qwen-style NextN/MTP predictor. There is no sidecar drafter, no second model identity and no `--model-draft` lifecycle. The two files preserve the same logical tensor set, names, shapes and ordering. They do **not** claim bit-identical dequantized values, logits or generated text: the storage and dequantization contracts are different. ## Artifacts | Priority | Representation | File | Exact size | SHA-256 | |---:|---|---|---:|---| | 1 | Native `TQ2_0` | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | 8,785,215,776 bytes / 8.18 GiB | `480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20` | | 2 | Standard `Q2_K` | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | 10,491,466,016 bytes / 9.77 GiB | `850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410` | ### 1. Native `TQ2_0` — primary artifact Choose this file when the target runtime supports the native `TQ2_0` tensor type. - smallest artifact in this repository; - 480 tensors stored natively as `TQ2_0`; - embedded target and MTP paths in one GGUF; - preserves the native ternary storage contract; - requires runtime support for `qwen35`, `draft-mtp` and `TQ2_0`. ### 2. Standard `Q2_K` — compatibility artifact Choose this file when standard `Q2_K` tensor support is preferable to native `TQ2_0` support. - all 480 native ternary tensors are represented as ordinary `Q2_K`; - the 17 tensors that were already `Q2_K` remain `Q2_K`; - final inventory: 497 `Q2_K` tensors; - no private GGUF tensor type and no custom tensor-type patch; - larger than the native artifact because standard `Q2_K` carries its own block metadata; - still requires runtime support for the `qwen35` architecture and embedded `draft-mtp` execution. The `Q2_K` file is a deterministic representation transform, not a generic calibration-based post-training quantization pass. ## Which file should I use? | Requirement | Recommended file | |---|---| | Minimum repository artifact size | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | | Native ternary execution | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | | Runtime already supports `TQ2_0` | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | | Prefer standard `Q2_K` tensor storage | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | | Runtime lacks `TQ2_0` but supports `Q2_K` | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | | Need embedded MTP without a sidecar drafter | either file | | Need architecture-independent output equivalence between representations | neither; no such claim is made | ## Model graph ```text self-contained GGUF │ ├── qwen35 target │ ├── target token embedding │ ├── 64 transformer blocks │ ├── final normalization │ └── target output path │ └── embedded MTP / NextN path ├── dedicated MTP token embedding ├── 1 Qwen-style NextN predictor block ├── MTP normalization └── MTP output path ``` The ordinary target path and the speculative path resolve from the same file and the same target identity. | Deployment property | External drafter | This repository | |---|---:|---:| | Target files | 1 | 1 | | Drafter files | 1 | 0 | | Runtime model identities | 2 | 1 | | Independent target/drafter version skew | possible | structurally removed | | Separate drafter checksum | required | not applicable | | `--model-draft` / `-md` | required | not used | | Embedded `draft-mtp` path | not intrinsic | yes | ## Tensor inventories The files contain the same 866 logical tensors. The difference is the storage type of the 480 ternary tensors. ### Native `TQ2_0` ```text TQ2_0 480 Q2_K 17 F32 359 Q4_K 1 Q5_K 6 Q6_K 3 ------------- total 866 ``` ### Standard `Q2_K` ```text Q2_K 497 F32 359 Q4_K 1 Q5_K 6 Q6_K 3 ------------- total 866 ``` ## Q2_K representation contract The compatibility artifact converts each native ternary block into an ordinary 84-byte `Q2_K` block: ```text 16 bytes scale/min metadata 64 bytes packed 2-bit symbols 2 bytes fp16 d 2 bytes fp16 dmin --------- 84 bytes total ``` For the 480 converted tensors, the transform follows this contract: ```text packed Q2_K symbols = packed native ternary symbols group metadata = 0x00 when all 16 symbols in the logical group are the center symbol 0xFF otherwise Q2_K d = fp16(fp32(native_scale) / 15) Q2_K dmin = fp16(fp32(native_scale) / 15) ``` Consequences: - the packed discrete symbol identities are preserved by construction; - the output uses the standard `Q2_K` block layout; - the conversion is deterministic for a fixed source artifact; - the conversion does not claim floating-point equivalence with native `TQ2_0` dequantization; - downstream logits and generated text may differ between representations. This distinction is intentional: the `Q2_K` variant is a compatibility representation of the ternary code field, not an assertion that the two dequantizers are interchangeable. ## Runtime contract Use a `llama.cpp` revision that supports: - the `qwen35` architecture; - the embedded `draft-mtp` path; - `TQ2_0` for the primary artifact, or standard `Q2_K` for the compatibility artifact. The `Q2_K` variant removes the native tensor-type requirement. It does not remove the architecture or MTP-runtime requirements. Runtime option names can change across `llama.cpp` revisions. Confirm the available flags with: ```bash llama-cli --help | grep -E 'draft-mtp|spec-draft|model-draft' ``` ## Download ### Primary `TQ2_0` ```bash hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \ Ternary-Bonsai-27B-MTP-TQ2_0.gguf \ SHA256SUMS \ --local-dir . ``` ### Compatibility `Q2_K` ```bash hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \ Ternary-Bonsai-27B-MTP-Q2_K.gguf \ SHA256SUMS \ --local-dir . ``` Verify either or both downloaded artifacts: ```bash sha256sum -c SHA256SUMS --ignore-missing ``` ## Inference Select one artifact: ```bash MODEL="Ternary-Bonsai-27B-MTP-TQ2_0.gguf" # or: # MODEL="Ternary-Bonsai-27B-MTP-Q2_K.gguf" ``` ### Ordinary target decode ```bash llama-cli \ -m "$MODEL" \ -p "Derive a cost model for speculative decoding." \ -n 256 ``` ### Embedded MTP ```bash llama-cli \ -m "$MODEL" \ --spec-type draft-mtp \ --spec-draft-n-max 2 \ -p "Derive a cost model for speculative decoding." \ -n 256 ``` There is deliberately no `-md` / `--model-draft` argument. ### OpenAI-compatible server ```bash llama-server \ -m "$MODEL" \ --spec-type draft-mtp \ --spec-draft-n-max 2 \ --host 127.0.0.1 \ --port 8080 ``` ## Speculative-decoding economics Embedded MTP is beneficial only when the accepted speculative work amortizes predictor and verification cost: ```text (C_draft + C_verify) / E[committed draft tokens] < C_target-step ``` Acceptance rate is an intermediate statistic, not the optimization objective. The relevant endpoint is end-to-end committed-token throughput under a fixed workload and configuration. A minimal controlled comparison is: ```text A. ordinary target decode B. draft-mtp with n_max = 1 C. draft-mtp with n_max = 2 ``` Hold constant: - exact model file and checksum; - prompt corpus and prompt order; - context length; - sampler chain and sampler parameters; - random seed where supported; - batch and micro-batch sizes; - thread count and affinity; - GPU-layer split and device placement; - KV-cache types; - warm-up policy; - output token budget. Report separately: - prompt-evaluation throughput; - committed generation throughput; - attempted and accepted draft tokens; - acceptance ratio; - time to first token; - peak host memory and device memory; - full command line and runtime revision. Do not compare the native and compatibility representations while attributing the entire difference to MTP: that changes two independent variables at once. No hardware-independent speedup claim is made by this repository. ## Portable artifact validation The public claims below are properties of the files and metadata, not of a specific workstation. | Check | Native `TQ2_0` | Standard `Q2_K` | |---|---:|---:| | Exact byte length recorded | pass | pass | | SHA-256 recorded | pass | pass | | GGUF V3 parse | pass | pass | | Architecture metadata: `qwen35` | pass | pass | | Logical tensor count: 866 | pass | pass | | Target depth: 64 blocks | pass | pass | | Embedded NextN depth: 1 block | pass | pass | | Tensor-name, shape and order inventory | pass | pass | | Expected type histogram | pass | pass | | Converted native tensors | not applicable | 480 / 480 | | Public metadata privacy scan | pass | pass | | Hub object size and content hash verification | pass | verified at publication | The model card intentionally publishes no workstation-specific throughput, device name, driver name, local path, hostname or private build log. ## Integrity The checksum is the authoritative artifact identity; the filename is a human-readable label. | File | SHA-256 | |---|---| | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | `480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20` | | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | `850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410` | Verify explicitly: ```bash printf '%s %s\n' \ '480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20' \ 'Ternary-Bonsai-27B-MTP-TQ2_0.gguf' \ | sha256sum -c - printf '%s %s\n' \ '850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410' \ 'Ternary-Bonsai-27B-MTP-Q2_K.gguf' \ | sha256sum -c - ``` ## Construction and provenance This is a model-engineering artifact. It is not a new pretraining run and not an instruction fine-tune. The final graph combines: - a pinned Ternary Bonsai 27B target trunk; - a Qwen3.6-derived NextN/MTP donor package; - a dedicated MTP embedding and output path; - a stock-compatible `qwen35` GGUF graph; - two published storage representations of the same logical tensor graph. | Role | Immutable source | Revision | Source SHA-256 | |---|---|---|---| | Ternary target trunk | [`prism-ml/Ternary-Bonsai-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf) | `3f8cc399dde45ac0475d023634974407af34907c` | `f659ca3dd7e28ada5d8b5f3637862d0d51ef433bde032ec4c8990ed27c91a385` | | MTP donor package | [`lym00/Qwen3.6-27B-MTP-ONLY-GGUF`](https://huggingface.co/lym00/Qwen3.6-27B-MTP-ONLY-GGUF) | `03b35db648da71e23ecfb239f70661e069b16054` | `97697fc5278d4bfd0afaa733ba28c4338cb0396b98d5f5acb781e515f72b52c0` | The donor package identifies Qwen3.6-derived MTP material but does not pin a separate underlying Qwen source revision. This card does not invent one. The predictor was grafted from a compatible donor. It was not jointly trained with the final ternary target. See `NOTICE` for attribution and `LICENSE` for terms. ## Scope of claims This repository claims: - a self-contained GGUF graph with target and embedded MTP paths; - exact artifact sizes and cryptographic identities; - the documented tensor inventories and graph structure; - complete conversion coverage for the 480 native ternary tensors in the `Q2_K` artifact; - standard `Q2_K` storage for the compatibility representation. This repository does **not** claim: - a new pretrained or fine-tuned model; - joint training of the final target and predictor; - output equivalence between `TQ2_0` and `Q2_K`; - preservation of upstream benchmark scores after graph construction or quantization; - a universal MTP speedup; - compatibility with every historical or future runtime revision; - multimodal capability or inclusion of a vision projector; - endorsement by Prism ML, Qwen, Alibaba Cloud, `lym00` or `llama.cpp`. ## Known limitations - Aggressive low-bit representations can affect accuracy, calibration and long-context behavior. - The grafted predictor may be suboptimal relative to a predictor jointly trained against the final target. - Speculative decoding can improve, match or reduce throughput depending on acceptance, backend, context, sampler and device placement. - The `Q2_K` compatibility representation is larger than native `TQ2_0`. - A successful parse and exact tensor inventory do not substitute for task-specific evaluation. - Reproducible performance evaluation requires publishing the complete runtime and workload configuration. ## License and attribution Repository artifacts are distributed under Apache-2.0, subject to the included `LICENSE` and `NOTICE` files and the terms of the source artifacts. This independent release is not endorsed by Prism ML, Qwen, Alibaba Cloud, `lym00` or the `llama.cpp` project. ## Citation ```bibtex @misc{vinpix2026ternarybonsai27bembeddedmtp, title = {Ternary Bonsai 27B with Embedded MTP: Native TQ2_0 and Standard Q2_K GGUF}, author = {vinpix}, year = {2026}, url = {https://huggingface.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF} } ``` --- **Primary representation:** native `TQ2_0` **Compatibility representation:** standard `Q2_K` **Deployment boundary:** one GGUF, one checksum, no sidecar drafter