Publish expert model card and update SHA256SUMS

README.md

@@ -17,226 +17,449 @@ tags:
   - local-llm
   - self-contained
 ---
+- q2_k
 
-<div align="center">
+# Ternary Bonsai 27B with Embedded MTP — GGUF
 
-# Ternary Bonsai 27B / Embedded MTP
+> **One target, one embedded NextN predictor, one deployment artifact per representation.**
 
-### One immutable GGUF. A ternary 27B-class target. An embedded Qwen-style NextN predictor. Stock `llama.cpp`.
+`qwen35` · 64 target blocks · 1 embedded NextN block · 866 tensors · Apache-2.0
 
-**8.18 GiB · GGUF V3 · 866 tensors · 64 target blocks + 1 NextN block**
+---
 
-`ordinary decode` &nbsp;·&nbsp; `draft-mtp` &nbsp;·&nbsp; no sidecar drafter &nbsp;·&nbsp; no private runtime patch
+## Release summary
 
-</div>
+This repository publishes two self-contained GGUF representations of the same
+logical model graph:
 
----
+1. **`TQ2_0` — primary, native ternary representation**
+2. **`Q2_K` — compatibility-oriented standard representation**
+
+Each file contains both the 27B-class target and its embedded Qwen-style
+NextN/MTP predictor. There is no sidecar drafter, no second model identity and
+no `--model-draft` lifecycle.
+
+The two files preserve the same logical tensor set, names, shapes and ordering.
+They do **not** claim bit-identical dequantized values, logits or generated text:
+the storage and dequantization contracts are different.
+
+## Artifacts
+
+| Priority | Representation | File | Exact size | SHA-256 |
+|---:|---|---|---:|---|
+| 1 | Native `TQ2_0` | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | 8,785,215,776 bytes / 8.18 GiB | `480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20` |
+| 2 | Standard `Q2_K` | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | 10,491,466,016 bytes / 9.77 GiB | `850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410` |
+
+### 1. Native `TQ2_0` — primary artifact
 
-> **The contribution is the artifact boundary.**  
-> The Ternary Bonsai target and the predictor consumed by `llama.cpp`'s `draft-mtp` path live in the same content-addressed GGUF.
+Choose this file when the target runtime supports the native `TQ2_0` tensor
+type.
 
-## Architecture
+- smallest artifact in this repository;
+- 480 tensors stored natively as `TQ2_0`;
+- embedded target and MTP paths in one GGUF;
+- preserves the native ternary storage contract;
+- requires runtime support for `qwen35`, `draft-mtp` and `TQ2_0`.
+
+### 2. Standard `Q2_K` — compatibility artifact
+
+Choose this file when standard `Q2_K` tensor support is preferable to native
+`TQ2_0` support.
+
+- all 480 native ternary tensors are represented as ordinary `Q2_K`;
+- the 17 tensors that were already `Q2_K` remain `Q2_K`;
+- final inventory: 497 `Q2_K` tensors;
+- no private GGUF tensor type and no custom tensor-type patch;
+- larger than the native artifact because standard `Q2_K` carries its own
+  block metadata;
+- still requires runtime support for the `qwen35` architecture and embedded
+  `draft-mtp` execution.
+
+The `Q2_K` file is a deterministic representation transform, not a generic
+calibration-based post-training quantization pass.
+
+## Which file should I use?
+
+| Requirement | Recommended file |
+|---|---|
+| Minimum repository artifact size | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` |
+| Native ternary execution | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` |
+| Runtime already supports `TQ2_0` | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` |
+| Prefer standard `Q2_K` tensor storage | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` |
+| Runtime lacks `TQ2_0` but supports `Q2_K` | `Ternary-Bonsai-27B-MTP-Q2_K.gguf` |
+| Need embedded MTP without a sidecar drafter | either file |
+| Need architecture-independent output equivalence between representations | neither; no such claim is made |
+
+## Model graph
 
 ```text
-Ternary-Bonsai-27B-MTP-TQ2_0.gguf
+self-contained GGUF
 │
 ├── qwen35 target
-│   └── 64 ternary transformer blocks
+│   ├── target token embedding
+│   ├── 64 transformer blocks
+│   ├── final normalization
+│   └── target output path
 │
-└── embedded speculative path
+└── embedded MTP / NextN path
     ├── dedicated MTP token embedding
     ├── 1 Qwen-style NextN predictor block
+    ├── MTP normalization
     └── MTP output path
 ```
 
-The ordinary and speculative execution paths resolve from the same file and the same target identity.
+The ordinary target path and the speculative path resolve from the same file
+and the same target identity.
 
-| Deployment property | External-drafter stack | This release |
+| Deployment property | External drafter | This repository |
 |---|---:|---:|
-| Target artifact | separate | embedded |
-| Predictor artifact | separate | embedded |
+| Target files | 1 | 1 |
+| Drafter files | 1 | 0 |
 | Runtime model identities | 2 | 1 |
-| Target/drafter version skew | possible | structurally eliminated |
-| Draft-model argument | required | not required |
-| Stock `qwen35` / `draft-mtp` path | not intrinsic | yes |
+| Independent target/drafter version skew | possible | structurally removed |
+| Separate drafter checksum | required | not applicable |
+| `--model-draft` / `-md` | required | not used |
+| Embedded `draft-mtp` path | not intrinsic | yes |
 
-This is not a replacement for a purpose-trained external drafter architecture. It is a different systems trade-off: an atomic deployment object with a single checksum, a single provenance chain and no sidecar lifecycle.
+## Tensor inventories
 
-## Artifact fingerprint
+The files contain the same 866 logical tensors. The difference is the storage
+type of the 480 ternary tensors.
 
-| Field | Value |
-|---|---|
-| File | `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` |
-| Size | 8,785,215,776 bytes / 8.18 GiB |
-| SHA-256 | `480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20` |
-| GGUF | V3 |
-| Architecture | `qwen35` |
-| Tensor count | 866 |
-| Target depth | 64 blocks |
-| Embedded NextN depth | 1 block |
-| Dominant weight type | `TQ2_0` |
-| Runtime contract | stock `llama.cpp` with `qwen35` and `draft-mtp` support |
-| License | Apache-2.0 |
+### Native `TQ2_0`
 
 ```text
 TQ2_0  480
-F32    359
 Q2_K    17
+F32    359
+Q4_K     1
 Q5_K     6
 Q6_K     3
+-------------
+total   866
+```
+
+### Standard `Q2_K`
+
+```text
+Q2_K   497
+F32    359
 Q4_K     1
+Q5_K     6
+Q6_K     3
 -------------
 total   866
 ```
 
-`TQ2_0` is the dominant target representation. Selected high-sensitivity and auxiliary tensors remain in K-quant or floating-point formats; the filename does not imply uniform tensor typing.
+## Q2_K representation contract
+
+The compatibility artifact converts each native ternary block into an ordinary
+84-byte `Q2_K` block:
+
+```text
+16 bytes  scale/min metadata
+64 bytes  packed 2-bit symbols
+ 2 bytes  fp16 d
+ 2 bytes  fp16 dmin
+---------
+84 bytes  total
+```
+
+For the 480 converted tensors, the transform follows this contract:
+
+```text
+packed Q2_K symbols = packed native ternary symbols
+
+group metadata =
+    0x00  when all 16 symbols in the logical group are the center symbol
+    0xFF  otherwise
+
+Q2_K d    = fp16(fp32(native_scale) / 15)
+Q2_K dmin = fp16(fp32(native_scale) / 15)
+```
+
+Consequences:
+
+- the packed discrete symbol identities are preserved by construction;
+- the output uses the standard `Q2_K` block layout;
+- the conversion is deterministic for a fixed source artifact;
+- the conversion does not claim floating-point equivalence with native
+  `TQ2_0` dequantization;
+- downstream logits and generated text may differ between representations.
+
+This distinction is intentional: the `Q2_K` variant is a compatibility
+representation of the ternary code field, not an assertion that the two
+dequantizers are interchangeable.
+
+## Runtime contract
+
+Use a `llama.cpp` revision that supports:
+
+- the `qwen35` architecture;
+- the embedded `draft-mtp` path;
+- `TQ2_0` for the primary artifact, or standard `Q2_K` for the compatibility
+  artifact.
+
+The `Q2_K` variant removes the native tensor-type requirement. It does not
+remove the architecture or MTP-runtime requirements.
+
+Runtime option names can change across `llama.cpp` revisions. Confirm the
+available flags with:
+
+```bash
+llama-cli --help | grep -E 'draft-mtp|spec-draft|model-draft'
+```
+
+## Download
 
-## Execution
+### Primary `TQ2_0`
 
-Ordinary target decode:
+```bash
+hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
+  Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
+  SHA256SUMS \
+  --local-dir .
+```
+
+### Compatibility `Q2_K`
+
+```bash
+hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
+  Ternary-Bonsai-27B-MTP-Q2_K.gguf \
+  SHA256SUMS \
+  --local-dir .
+```
+
+Verify either or both downloaded artifacts:
+
+```bash
+sha256sum -c SHA256SUMS --ignore-missing
+```
+
+## Inference
+
+Select one artifact:
+
+```bash
+MODEL="Ternary-Bonsai-27B-MTP-TQ2_0.gguf"
+# or:
+# MODEL="Ternary-Bonsai-27B-MTP-Q2_K.gguf"
+```
+
+### Ordinary target decode
 
 ```bash
 llama-cli \
-  -m Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
-  -p "Derive the latency model for speculative decoding." \
+  -m "$MODEL" \
+  -p "Derive a cost model for speculative decoding." \
   -n 256
 ```
 
-Embedded MTP:
+### Embedded MTP
 
 ```bash
 llama-cli \
-  -m Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
+  -m "$MODEL" \
   --spec-type draft-mtp \
   --spec-draft-n-max 2 \
-  -p "Derive the latency model for speculative decoding." \
+  -p "Derive a cost model for speculative decoding." \
   -n 256
 ```
 
 There is deliberately no `-md` / `--model-draft` argument.
 
-## Speculative-decoding model
+### OpenAI-compatible server
+
+```bash
+llama-server \
+  -m "$MODEL" \
+  --spec-type draft-mtp \
+  --spec-draft-n-max 2 \
+  --host 127.0.0.1 \
+  --port 8080
+```
+
+## Speculative-decoding economics
 
-MTP is profitable only when accepted speculative work amortizes predictor and verification cost:
+Embedded MTP is beneficial only when the accepted speculative work amortizes
+predictor and verification cost:
 
 ```text
-(Cdraft + Cverify) / E[committed draft tokens] < Ctarget-step
+(C_draft + C_verify) / E[committed draft tokens] < C_target-step
 ```
 
-Acceptance rate is an intermediate statistic, not the objective. The relevant quantity is end-to-end committed-token throughput under an identical prompt distribution, sampler, context and device-placement policy.
+Acceptance rate is an intermediate statistic, not the optimization objective.
+The relevant endpoint is end-to-end committed-token throughput under a fixed
+workload and configuration.
 
-A disciplined sweep is small:
+A minimal controlled comparison is:
 
 ```text
-baseline
-draft-mtp, n_max = 1
-draft-mtp, n_max = 2
+A. ordinary target decode
+B. draft-mtp with n_max = 1
+C. draft-mtp with n_max = 2
 ```
 
-Retain MTP only where the complete pipeline wins. Larger draft depth is not intrinsically better, and no hardware-independent token/s claim is made here.
+Hold constant:
 
-## Construction
+- exact model file and checksum;
+- prompt corpus and prompt order;
+- context length;
+- sampler chain and sampler parameters;
+- random seed where supported;
+- batch and micro-batch sizes;
+- thread count and affinity;
+- GPU-layer split and device placement;
+- KV-cache types;
+- warm-up policy;
+- output token budget.
 
-This is a model-engineering artifact, not a new pretraining run and not an instruction fine-tune.
+Report separately:
 
-The final graph combines:
+- prompt-evaluation throughput;
+- committed generation throughput;
+- attempted and accepted draft tokens;
+- acceptance ratio;
+- time to first token;
+- peak host memory and device memory;
+- full command line and runtime revision.
 
-- a pinned Ternary Bonsai 27B trunk;
-- a Qwen3.6-derived NextN/MTP donor;
-- a dedicated MTP embedding and output path;
-- a stock-compatible `qwen35` GGUF layout;
-- a mixed-precision tensor policy centered on `TQ2_0`;
-- candidate selection gated by structural integrity, language-model quality and stock-runtime behavior.
+Do not compare the native and compatibility representations while attributing
+the entire difference to MTP: that changes two independent variables at once.
 
-The predictor was grafted from a compatible donor. It was **not jointly trained with the final ternary trunk**. The exact claim is therefore functional embedded speculation in one stock-compatible artifact, not co-trained target/predictor optimality.
+No hardware-independent speedup claim is made by this repository.
 
-## Validation envelope
+## Portable artifact validation
 
-The release was qualified on properties that travel with the artifact:
+The public claims below are properties of the files and metadata, not of a
+specific workstation.
 
-| Check | Result |
-|---|---:|
-| Exact public byte length and SHA-256 | pass |
-| GGUF V3 parse | pass |
-| `qwen35` architecture | pass |
-| Exact 866-tensor inventory | pass |
-| 64 target blocks + 1 NextN block | pass |
-| Ordinary generation from the released file | pass |
-| `draft-mtp` generation from the same file | pass |
-| Requested generation budget completed | pass |
-| Stock-runtime compatibility matrix | 6 / 6 |
-| Unmodified `llama.cpp` revisions exercised | 2 |
-| Full public binary privacy scan | pass |
-| Remote Hugging Face size and LFS SHA-256 match | pass |
+| Check | Native `TQ2_0` | Standard `Q2_K` |
+|---|---:|---:|
+| Exact byte length recorded | pass | pass |
+| SHA-256 recorded | pass | pass |
+| GGUF V3 parse | pass | pass |
+| Architecture metadata: `qwen35` | pass | pass |
+| Logical tensor count: 866 | pass | pass |
+| Target depth: 64 blocks | pass | pass |
+| Embedded NextN depth: 1 block | pass | pass |
+| Tensor-name, shape and order inventory | pass | pass |
+| Expected type histogram | pass | pass |
+| Converted native tensors | not applicable | 480 / 480 |
+| Public metadata privacy scan | pass | pass |
+| Hub object size and content hash verification | pass | verified at publication |
+
+The model card intentionally publishes no workstation-specific throughput,
+device name, driver name, local path, hostname or private build log.
+
+## Integrity
+
+The checksum is the authoritative artifact identity; the filename is a
+human-readable label.
+
+| File | SHA-256 |
+|---|---|
+| `Ternary-Bonsai-27B-MTP-TQ2_0.gguf` | `480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20` |
+| `Ternary-Bonsai-27B-MTP-Q2_K.gguf` | `850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410` |
+
+Verify explicitly:
 
-Stock compatibility points:
+```bash
+printf '%s  %s\n' \
+  '480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20' \
+  'Ternary-Bonsai-27B-MTP-TQ2_0.gguf' \
+  | sha256sum -c -
+
+printf '%s  %s\n' \
+  '850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410' \
+  'Ternary-Bonsai-27B-MTP-Q2_K.gguf' \
+  | sha256sum -c -
+```
 
-- `f955e394bf94e01e5e36186d13c985727e5ef5b5`
-- `ae735b13148db250911ce9d07663d1b95474e7b0`
+## Construction and provenance
 
-These are tested points, not a statement about every historical or future revision.
+This is a model-engineering artifact. It is not a new pretraining run and not
+an instruction fine-tune.
 
-## Provenance
+The final graph combines:
+
+- a pinned Ternary Bonsai 27B target trunk;
+- a Qwen3.6-derived NextN/MTP donor package;
+- a dedicated MTP embedding and output path;
+- a stock-compatible `qwen35` GGUF graph;
+- two published storage representations of the same logical tensor graph.
 
 | Role | Immutable source | Revision | Source SHA-256 |
 |---|---|---|---|
-| Ternary target trunk | [`prism-ml/Ternary-Bonsai-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf/blob/3f8cc399dde45ac0475d023634974407af34907c/Ternary-Bonsai-27B-F16.gguf) | `3f8cc399dde45ac0475d023634974407af34907c` | `f659ca3dd7e28ada5d8b5f3637862d0d51ef433bde032ec4c8990ed27c91a385` |
-| MTP donor package | [`lym00/Qwen3.6-27B-MTP-ONLY-GGUF`](https://huggingface.co/lym00/Qwen3.6-27B-MTP-ONLY-GGUF/blob/03b35db648da71e23ecfb239f70661e069b16054/Qwen3.6-27B-MTP-bf16.gguf) | `03b35db648da71e23ecfb239f70661e069b16054` | `97697fc5278d4bfd0afaa733ba28c4338cb0396b98d5f5acb781e515f72b52c0` |
-
-The donor package identifies Qwen3.6-derived MTP material but does not pin a separate underlying Qwen source revision. This card does not invent one.
+| Ternary target trunk | [`prism-ml/Ternary-Bonsai-27B-gguf`](https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf) | `3f8cc399dde45ac0475d023634974407af34907c` | `f659ca3dd7e28ada5d8b5f3637862d0d51ef433bde032ec4c8990ed27c91a385` |
+| MTP donor package | [`lym00/Qwen3.6-27B-MTP-ONLY-GGUF`](https://huggingface.co/lym00/Qwen3.6-27B-MTP-ONLY-GGUF) | `03b35db648da71e23ecfb239f70661e069b16054` | `97697fc5278d4bfd0afaa733ba28c4338cb0396b98d5f5acb781e515f72b52c0` |
 
-See [`NOTICE`](./NOTICE) for attribution and [`LICENSE`](./LICENSE) for terms.
+The donor package identifies Qwen3.6-derived MTP material but does not pin a
+separate underlying Qwen source revision. This card does not invent one.
 
-## Public artifact boundary
+The predictor was grafted from a compatible donor. It was not jointly trained
+with the final ternary target.
 
-Before publication, two path-valued quantization metadata fields were sanitized:
+See `NOTICE` for attribution and `LICENSE` for terms.
 
-- `quantize.imatrix.file`
-- `quantize.imatrix.dataset`
+## Scope of claims
 
-Only those metadata strings changed. Tensor payloads, dimensions, offsets, topology and file length did not.
+This repository claims:
 
-The checksum below is the authoritative identity of the public artifact.
+- a self-contained GGUF graph with target and embedded MTP paths;
+- exact artifact sizes and cryptographic identities;
+- the documented tensor inventories and graph structure;
+- complete conversion coverage for the 480 native ternary tensors in the
+  `Q2_K` artifact;
+- standard `Q2_K` storage for the compatibility representation.
 
-## Technical boundaries
+This repository does **not** claim:
 
-- The predictor is grafted, not jointly trained with the final target.
-- Aggressive quantization can alter accuracy, calibration and long-context behavior.
-- Upstream Bonsai benchmark results are not asserted as measurements of this file.
-- MTP may improve, match or reduce throughput depending on backend and workload.
-- The repository contains a text-generation GGUF, not a multimodal projector.
-- The release is independent and is not endorsed by Prism ML, Qwen, Alibaba Cloud, `lym00` or `llama.cpp`.
+- a new pretrained or fine-tuned model;
+- joint training of the final target and predictor;
+- output equivalence between `TQ2_0` and `Q2_K`;
+- preservation of upstream benchmark scores after graph construction or
+  quantization;
+- a universal MTP speedup;
+- compatibility with every historical or future runtime revision;
+- multimodal capability or inclusion of a vision projector;
+- endorsement by Prism ML, Qwen, Alibaba Cloud, `lym00` or `llama.cpp`.
 
-## Verify
+## Known limitations
 
-```bash
-hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
-  Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
-  SHA256SUMS \
-  --local-dir .
+- Aggressive low-bit representations can affect accuracy, calibration and
+  long-context behavior.
+- The grafted predictor may be suboptimal relative to a predictor jointly
+  trained against the final target.
+- Speculative decoding can improve, match or reduce throughput depending on
+  acceptance, backend, context, sampler and device placement.
+- The `Q2_K` compatibility representation is larger than native `TQ2_0`.
+- A successful parse and exact tensor inventory do not substitute for
+  task-specific evaluation.
+- Reproducible performance evaluation requires publishing the complete runtime
+  and workload configuration.
 
-sha256sum -c SHA256SUMS
-```
+## License and attribution
 
-Expected digest:
+Repository artifacts are distributed under Apache-2.0, subject to the included
+`LICENSE` and `NOTICE` files and the terms of the source artifacts.
 
-```text
-480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20
-```
+This independent release is not endorsed by Prism ML, Qwen, Alibaba Cloud,
+`lym00` or the `llama.cpp` project.
 
 ## Citation
 
 ```bibtex
-@misc{vinpix2026ternarybonsai27bmtp,
-  title  = {Ternary Bonsai 27B with Embedded MTP: A One-File GGUF for llama.cpp},
+@misc{vinpix2026ternarybonsai27bembeddedmtp,
+  title  = {Ternary Bonsai 27B with Embedded MTP: Native TQ2_0 and Standard Q2_K GGUF},
   author = {vinpix},
   year   = {2026},
   url    = {https://huggingface.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF}
 }
 ```
 
-<div align="center">
-
-**Ternary target · Embedded predictor · One artifact boundary**
+---
 
-</div>
+**Primary representation:** native `TQ2_0`  
+**Compatibility representation:** standard `Q2_K`  
+**Deployment boundary:** one GGUF, one checksum, no sidecar drafter

SHA256SUMS

@@ -1 +1,2 @@
 480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20  Ternary-Bonsai-27B-MTP-TQ2_0.gguf
+850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410  Ternary-Bonsai-27B-MTP-Q2_K.gguf