--- library_name: mlx license: mit pipeline_tag: image-text-to-text tags: - mlx - apple-silicon - speculative-decoding - qwen - qwen3 - qwen3-next - mtp - mtplx - local-ai - vision-language - multimodal base_model: - wang-yang/Ornith-1.0-35B-MTPLX - deepreinforce-ai/Ornith-1.0-35B --- # Ornith-1.0-35B-MTPLX-Vision A unified local model that both codes at the Ornith-1.0-35B level and sees images/videos, packaged for Apple Silicon via **MTPLX** native speculative decoding with multi-token-prediction (MTP). This repository restores the multimodal vision tower that is typically stripped in text-only MLX/MTPLX conversions. ## Architecture - **Body**: 4-bit affine, group size 64 (from [wang-yang/Ornith-1.0-35B-MTPLX](https://huggingface.co/wang-yang/Ornith-1.0-35B-MTPLX)) - **MTP sidecar (mtp.safetensors)**: 4-bit quantized MTP heads. - **Vision tower (vision_tower.safetensors, 333 tensors, bf16 unquantized)**: Extracted verbatim from the official checkpoint's own `model.visual.*` weights (shard 16 of [deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B)) and renamed to the `vision_tower.*` prefix for MTPLX native VLM compatibility (~852 MB). ## Usage (MTPLX) ```bash mtplx start --model --port 8092 \ --chat-template-path /chat_template.jinja ``` Serves an OpenAI-compatible endpoint supporting both text and image input payloads via `POST /v1/chat/completions`. The MTPLX CLI defaults to `mtp_history_policy=committed`, which this model requires for healthy deep-position acceptance. For best throughput on short prompts also pass `--draft-temperature 0` (greedy draft). ## Performance (measured) Apple M5 Pro 64 GB, MTPLX 1.0.4, temperature 0.6, `mtp_history_policy=committed`, `--draft-temperature 0`, thinking OFF, warm (4 warmup prompts discarded), 8 measured prompts from the calibration_coding suite, max_tokens=192. Median over 3 independent runs. | Depth | tok/s e2e (range) | tok/s decode (range) | speedup vs AR e2e (range) | acceptance pos1/2/3 | |---|---|---|---|---| | AR (`--no-mtp`) | 35-60 | 36-63 | 1.00x | - | | MTP depth 1 | 51-64 | 53-71 | 1.07-1.44x | 0.873 | | MTP depth 2 | 50-68 | 49-73 | 1.14-1.69x | 0.881 / 0.691 | | MTP depth 3 | 43-62 | 43-66 | 1.04-1.64x | 0.850 / 0.708 / 0.515 | Two throughput definitions are reported because they answer different questions: - **tok/s (wall-clock e2e)** = `generated_tokens / total_elapsed`, includes prompt prefill. The real end-to-end speed a user experiences and the honest denominator for the speedup ratio. - **tok/s (decode-only)** = `generated_tokens / decode_elapsed`, excludes prefill. Comparable to the older 2026-06-27 M3 Max figures that used decode-only throughput. **Acceptance is the stable MTP health signal.** Across 3 independent runs the per-position acceptance was rock-stable: pos1 0.85-0.87, pos2 0.69-0.71, pos3 0.52. Tok/s varies run-to-run (AR e2e bounces 35-60 tok/s depending on thermal/contention state), so the speedup ratio inherits that noise; the acceptance numbers are the reliable indicator that the MTP graft is healthy. The earlier README reported 1.06x; that number used a single cold trial with the non-default `mtp_history_policy=cycle`, which collapses pos3 acceptance to ~0.27. With the committed history policy that the CLI already serves by default, deep acceptance recovers and MTP depth 2/3 win clearly over AR. ### Why this vision graft works (and the non-vision source does not) The non-vision source ([wang-yang/Ornith-1.0-35B-MTPLX](https://huggingface.co/wang-yang/Ornith-1.0-35B-MTPLX)) collapses to pos1 ~= 0.23 acceptance on M5 Pro under every history policy tested (committed, cycle, full, last_window, with and without greedy draft). This is not a policy or loader bug. A direct weight diff shows the cause: - The MTP draft head (`mtp.safetensors`) is **bit-for-bit identical** between wang-yang and this vision variant (46/46 leaf tensors, 0 differences). - The MTP head's own gate quantization is 4-bit in both. - The target body, however, **differs in 264 of 1803 leaf tensors** (concentrated in `linear_attn`, MoE gates, `shared_expert`, `switch_mlp` across layers 11-33). The two models are not the same weights with a vision tower bolted on; the vision variant's body was re-derived such that the shipped draft head matches it, while wang-yang's body does not. So the published wang-yang draft head diverges from wang-yang's own target body, and MTP acceptance collapses. This vision variant pairs a body whose weights match the shipped draft head. The vision tower itself is lazy-loaded (only when a request carries an image), so text-only throughput is unaffected by it. ### A higher-acceptance alternative For the best MTP acceptance on this family, the [Shiftedx/ornith-1.0-35b-abliterated-mxfp4-vision-mtplx](https://huggingface.co/Shiftedx/ornith-1.0-35b-abliterated-mxfp4-vision-mtplx) variant (mxfp4 body, 5-bit MTP head) measures, on the same M5 Pro 64GB under the same committed+greedy config: | Depth | tok/s (e2e) | speedup vs AR | acceptance pos1/2/3 | |---|---|---|---| | AR | 67.6 | 1.00x | - | | 3 | 87.4 | 1.29x | 0.93 / 0.86 / 0.76 | Its pos3 acceptance (0.76) is the highest of the Ornith MTP variants tested. Note: loading that variant on stock MTPLX <= 2.1.0 requires a one-line fix to `_infer_prequantized_group_size` (it miscomputes the group size for 5-bit MTP heads); see [MTPLX issue #182](https://github.com/youssofal/MTPLX/issues/182) / [PR #183](https://github.com/youssofal/MTPLX/pull/183). Numbers are indicative, not a benchmark suite. ## Sources And Attribution | Component | Source | License | |---|---|---| | Base model | [deepreinforce-ai/Ornith-1.0-35B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B) | MIT | | MTP source | [Qwen/Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) | Apache-2.0 | | Quantized Text/MTP repo | [wang-yang/Ornith-1.0-35B-MTPLX](https://huggingface.co/wang-yang/Ornith-1.0-35B-MTPLX) | MIT / Apache-2.0 | | MTPLX conversion | [youssofal/mtplx](https://github.com/youssofal/MTPLX) | Apache-2.0 | This derivative is released under **MIT**, preserving the base model's license. The grafted MTP tensors originate from the Apache-2.0 licensed Qwen3.5-35B-A3B; that license and its NOTICE apply to those tensors.