| --- |
| license: apache-2.0 |
| base_model: tencent/Hy3 |
| library_name: mlx |
| pipeline_tag: text-generation |
| language: en |
| tags: |
| - mlx |
| - apple-silicon |
| - hy3 |
| - mixture-of-experts |
| - mtp |
| - speculative-decoding |
| - mtplx |
| --- |
| |
| # Hy3-Demolition-MLX lite-v1-mtp |
|
|
| [Explore the model guide](https://huggingface.co/spaces/philipjohnbasile/local-ai-guide) · [All public work](https://huggingface.co/philipjohnbasile) |
|
|
| ## Release at a glance |
|
|
| | | This artifact | |
| |---|---| |
| | Purpose | The Lite trunk with Hy3's NextN sidecar restored for MTP experiments. | |
| | Runtime | Requires a compatible Hy3 model implementation and MTPLX runtime contract. The upstream backend shipped, but that does not qualify this artifact. | |
| | Status | Runtime verification pending; see the evidence and limits below. | |
| | Tensor download | 112.57 GB (104.84 GiB) of root `.safetensors` files, including any root sidecars. This is a file-size total, not peak RAM. | |
| | Read first | Use the AR sibling for the documented generation path. No current end-to-end MTP speedup is established here. | |
|
|
| The **MTP-equipped** variant of [lite-v1](https://huggingface.co/philipjohnbasile/hy3-demolition-mlx-lite-v1): its fused trunk with the Hy3 **NextN (Multi-Token-Prediction) sidecar** grafted back on (`num_nextn_predict_layers=1`, num_experts=192), for self-speculative decoding on MTPLX. |
| |
| ## Runtime status — reviewed September 10, 2026 |
| |
| The original `recognized-backend-pending` observation came from MTPLX |
| 2.0.1. It is historical: the maintainer subsequently confirmed that the |
| Hy3 and Qwen MTP backend work shipped in 2.1.0 through the release branch. |
| [Read the upstream shipping record](https://github.com/youssofal/MTPLX/pull/142). |
| |
| That code shipment does not qualify this exact checkpoint. The source |
| Hy3 implementation, runtime contract, loading, output agreement, and |
| performance still need to be checked together. No fresh end-to-end MTP |
| qualification of this artifact is claimed here. Upstream mlx-lm PR |
| [#1211](https://github.com/ml-explore/mlx-lm/pull/1211) remained open at review. |
| |
| The documented AR path is the [sibling model](https://huggingface.co/philipjohnbasile/hy3-demolition-mlx-lite-v1). |
| |
| ## How it was built |
| |
| The MTP head consumes the trunk's final hidden state (hidden_size 4096, unchanged by pruning) and runs its own MoE on the global `num_experts`. So the base checkpoint's `mtp.*` sidecar grafts directly onto the fused AR trunk — no re-heal, no re-prune of the trunk. |
|
|
| Graft script + `mtplx inspect` receipts: https://github.com/PhilipJohnBasile/hy3-demolition-mlx (`scripts/38_mtp_sidecar_graft.py`, `eval/receipts/mtplx_inspect_*.json`). |
|
|
| ## Limitations |
|
|
| - **Does not run on stock mlx_lm as MTP.** The fast MTP path needs the MTPLX backend; mlx-lm's own per-token self-speculative loop is ~4.7× *slower* than AR (measured), which is why the MTPLX batched-verify backend is the target. |
| - Everything from the base lite-v1 card applies (quantized MoE, English/agent focus, no tool execution). |
| - End-to-end MTP behavior is **unverified** until the backend loads it; recognition is structural (`mtplx inspect`), not a live run. |
| |
| Base recipe + receipts: https://github.com/PhilipJohnBasile/hy3-demolition-mlx |
| |
| ## Other serving applications |
| |
| No current LM Studio or Ollama qualification of this artifact is recorded |
| here. An architecture becoming available in one library does not establish |
| support in every application. Use the AR sibling's pinned release recipe |
| as the documented starting point. |
| |
| ## Experimental streaming |
| |
| The source project's SSD pager measurements concern an AR serving path. |
| They do not qualify this MTP variant on a smaller-memory machine. |
| [Read the AR streaming experiment](https://github.com/PhilipJohnBasile/hy3-demolition-mlx/blob/main/docs/64gb-feasibility.md). |
| |
| |
