README.md

---
license: apache-2.0
base_model: empero-ai/Qwythos-9B-v2
language:
- en
library_name: mlx
pipeline_tag: image-text-to-text
tags:
- mlx
- mlx-vlm
- safetensors
- bfloat16
- qwen3_5
- qwen3.5
- qwythos
- reasoning
- long-context
- 1M-context
- multimodal
- ftpo
- function-calling
- tool-use
- agentic
- cybersecurity
- biomedical
- conversational
---

# Qwythos-9B-v2-MLX-bf16

This repository, **xunkutech-ai/Qwythos-9B-v2-MLX-bf16**, contains a
bfloat16 MLX conversion of
[empero-ai/Qwythos-9B-v2](https://huggingface.co/empero-ai/Qwythos-9B-v2)
for Apple Silicon inference with MLX-VLM and applications that use compatible
MLX backends.

No additional fine-tuning was performed for this repository. The weights were
converted from the new upstream v2 checkpoint to MLX-compatible safetensors
while preserving the upstream Apache-2.0 license and model behavior.

## v2 Upstream Notice

This conversion is built from the newer **Qwythos-9B-v2** release, not from
the earlier Qwythos-9B-Claude-Mythos-5-1M checkpoint.

According to the upstream model card, v2 is a robustness-focused update that:

- eliminates the repetition and degeneration observed under greedy or
  low-temperature decoding;
- preserves the reasoning capability of the earlier Qwythos release;
- restores the native Qwen3.5 Multi-Token Prediction (MTP) head in the
  upstream checkpoint;
- removes unsolicited identity preambles;
- retains the 1,048,576-token YaRN configuration and the Qwen3.5
  multimodal-capable architecture;
- remains intentionally less refusal-oriented for research, cybersecurity,
  red-teaming, biology, chemistry, pharmacology, and clinical topics.

The upstream developers describe v2 as a hygiene and robustness release rather
than a new capability tier.

## Model Summary

- **Repository:** xunkutech-ai/Qwythos-9B-v2-MLX-bf16
- **Format:** MLX safetensors
- **Precision:** bfloat16
- **Parameters:** about 9B
- **Architecture:** Qwen3.5 hybrid vision-language model
- **Text layers:** 32
- **Vision layers:** 27
- **Configured context length:** 1,048,576 tokens with YaRN factor 4
- **Original YaRN window:** 262,144 tokens
- **Primary use:** local text-first reasoning and visual-language inference on
  Apple Silicon
- **Upstream model:** empero-ai/Qwythos-9B-v2
- **Upstream revision converted:**
  2178f73a9b5ea28ccd8f6096ef6bac5cd59c9d8b
- **MTP in this repository:** excluded from the main model; use the standalone
  MTP drafter described below

The model retains the Qwen3.5 visual tower and image/video special tokens.
However, the upstream card primarily reports text reasoning evaluations, so
users should evaluate visual performance for their own workloads.

## About the Upstream Model

Qwythos-9B-v2 is an Empero AI reasoning model built on Qwen3.5-9B. The v2
release applies **FTPO (Final-Token Preference Optimization)** to the earlier
Qwythos checkpoint. Instead of broadly changing the output distribution, FTPO
targets the token positions that begin repetition loops and trains the model to
prefer coherent alternatives.

The upstream card reports approximately 2,000 automatically mined preference
tuples, LoRA rank 256, alpha 128, a learning rate of 1.5e-5, and one training
epoch with early stopping. The MTP head was restored from the Qwen3.5-9B base;
the upstream card notes that it was not co-trained with the FTPO update.

### Upstream-Reported Evaluation Results

The following results are copied in summary form from the upstream model card.
They were measured with Empero AI's internal generative chain-of-thought
harness and should not be treated as independently reproduced by this
conversion.

| Benchmark | Upstream Qwythos-9B-v2 result |
|---|---:|
| MMLU, chain-of-thought | 83.8% |
| MMLU, 5-shot log-likelihood | 69.6% |
| ARC-Challenge | 96.4% |
| GPQA-Diamond | 49.0% |
| GSM8K | 93.6% |
| HumanEval pass@1 | 77.4% |
| Looping rate, greedy | 0.0% |
| Refusal rate | 0.0% |

See the
[upstream model card](https://huggingface.co/empero-ai/Qwythos-9B-v2)
for its complete evaluation methodology, comparisons, sample generations,
training details, and limitations.

## Install

~~~bash
pip install -U mlx-vlm
~~~

or:

~~~bash
uv pip install -U mlx-vlm
~~~

This is a Qwen3.5 vision-language checkpoint and should be loaded with
MLX-VLM. This card does not claim standalone MLX-LM compatibility.

## Quick Start: Text-Only Generation

MLX-VLM supports text-only prompts by omitting the image argument:

~~~bash
mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --prompt "Prove that there are infinitely many primes." \
  --max-tokens 2048 \
  --temperature 0.6 \
  --repetition-penalty 1.05 \
  --gen-kwargs '{"top_p": 0.95, "top_k": 20}'
~~~

For difficult reasoning, coding, tool-use, or long-context tasks, increase the
generation budget.

## Image Understanding

The converted checkpoint retains the upstream Qwen3.5 vision tower:

~~~bash
mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --image /path/to/image.jpg \
  --prompt "Describe this image and explain the important details." \
  --max-tokens 1024 \
  --temperature 0.6
~~~

Multiple image paths can be passed after the image argument when supported by
the installed MLX-VLM release.

## Python Example

~~~python
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "xunkutech-ai/Qwythos-9B-v2-MLX-bf16"

# Load the MLX model and multimodal processor.
model, processor = load(model_path)
config = load_config(model_path)

# Use a local path, URL, or PIL image supported by MLX-VLM.
images = ["/path/to/image.jpg"]
prompt = "Analyze this image step by step."

# Format the prompt with the correct number of visual inputs.
formatted_prompt = apply_chat_template(
    processor,
    config,
    prompt,
    num_images=len(images),
)

result = generate(
    model,
    processor,
    formatted_prompt,
    images,
    max_tokens=1024,
    temperature=0.6,
    verbose=False,
)

print(result.text)
~~~

## Recommended Sampling

The upstream model is designed to reason before answering. Its recommended
starting point is:

~~~python
generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 16384,
}
~~~

The upstream v2 card reports that repetition penalty is optional rather than
load-bearing because the looping behavior was trained out. A smaller
max_tokens value is usually sufficient for simple prompts; difficult reasoning
tasks benefit from a larger budget.

## Long Context

The model config contains a 1,048,576-token maximum position setting with
static YaRN factor 4 over an original 262,144-token window.

Practical usable context depends on unified memory, prompt modality, KV-cache
configuration, runtime version, and generation length. Static YaRN can also
carry a short-context quality trade-off. Test the intended context length on
the target machine rather than assuming the configured maximum is inexpensive
or equally reliable for every workload.

## Standalone MTP Drafter

The upstream v2 checkpoint restores 15 native MTP tensors. They are
intentionally excluded from this main repository, which contains 760 main-model
tensors. A companion MLX drafter can be stored separately as:

**xunkutech-ai/Qwythos-9B-v2-MLX-bf16-mtp-draft**

The standalone drafter uses model_type qwen3_5_mtp and contains only the
sanitized MTP weights plus tokenizer files. With an MLX-VLM release that
supports Qwen3.5 MTP speculative decoding, load it with:

~~~bash
mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --draft-model xunkutech-ai/Qwythos-9B-v2-MLX-bf16-mtp-draft \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "Explain speculative decoding with MTP." \
  --max-tokens 2048 \
  --temperature 0.6
~~~

The companion identifier can also be replaced with a local drafter directory.
Speculative-decoding speedup is workload- and hardware-dependent. The upstream
card cautions that acceptance may be modest because the restored MTP head was
not co-trained with the FTPO-updated main weights.

## Conversion Notes

### Conversion Provenance

- **Source:** empero-ai/Qwythos-9B-v2
- **Source revision:** 2178f73a9b5ea28ccd8f6096ef6bac5cd59c9d8b
- **Source weights:** one 19,306,305,296-byte safetensors file
- **Output dtype:** BF16
- **Output tensor count:** 760
- **Output weight data:** 18,819,627,488 bytes
- **Output shards:** 4
- **Maximum shard tensor data:** 5 GiB
- **Additional training:** none

The source checkpoint was converted with a streaming, GPU-independent
safetensors converter. Unchanged BF16 tensors were copied without a
full-model BF16-to-FP32 round trip. Only framework-required tensor
transformations were applied.

### Tensor Transformations

1. Hugging Face tensor names were remapped to MLX-VLM's Qwen3.5 namespace.
2. Qwen3.5 RMSNorm weights received the MLX-required +1 offset.
3. Linear-attention Conv1d weights were moved from
   [out, 1, kernel] to [out, kernel, 1].
4. The visual patch-embedding Conv3d weight was transposed from
   [1152, 3, 2, 16, 16] to MLX channels-last layout
   [1152, 2, 16, 16, 3].
5. The 15 upstream MTP tensors were omitted from this main-model variant and
   extracted separately for the companion drafter.

## Compatibility Fixes Applied

### 1. partial_rotary_factor Location

The upstream config stores partial_rotary_factor at the text_config top level,
while MLX-VLM Qwen3.5 code paths consume it from rope_parameters.

The converted config moves the value into:

~~~json
{
  "text_config": {
    "rope_parameters": {
      "rope_type": "yarn",
      "factor": 4.0,
      "original_max_position_embeddings": 262144,
      "mrope_section": [11, 11, 10],
      "rope_theta": 10000000,
      "partial_rotary_factor": 0.25
    }
  }
}
~~~

This avoids errors reporting that rope_parameters is missing
partial_rotary_factor.

### 2. vision_config.model_type

The upstream value qwen3_5_vision was normalized to qwen3_5 for compatibility
with MLX-VLM versions and bundled MLX backends that use a narrower model-type
whitelist.

### 3. Vision Conv3d Weight Layout

Hugging Face stores the visual patch projection in channels-first Conv3d
layout:

~~~text
[out, in, depth, height, width] = [1152, 3, 2, 16, 16]
~~~

MLX expects channels-last kernel layout:

~~~text
[out, depth, height, width, in] = [1152, 2, 16, 16, 3]
~~~

Both the safetensors shape and the underlying BF16 data were transposed. This
fix is required for model loading and downstream sensitivity measurement or
quantization; changing only the metadata shape would be incorrect.

## Verification

The conversion was independently checked without materializing the full model
in memory:

- all 760 tensors are unique and BF16;
- no MTP tensor is present in this main-model repository;
- the safetensors index exactly matches the shard contents;
- each shard contains at most 5 GiB of tensor data;
- partial_rotary_factor is present in rope_parameters;
- vision_config.model_type is qwen3_5;
- every BF16 element of the converted visual patch projection matches the
  expected source transpose.

## Files

| File | Size | Description |
|---|---:|---|
| config.json | 2.9 KB | MLX-compatible Qwen3.5 model configuration |
| model-00001-of-00004.safetensors | 5.28 GB | BF16 model shard 1/4 |
| model-00002-of-00004.safetensors | 5.36 GB | BF16 model shard 2/4 |
| model-00003-of-00004.safetensors | 5.34 GB | BF16 model shard 3/4 |
| model-00004-of-00004.safetensors | 2.84 GB | BF16 model shard 4/4 |
| model.safetensors.index.json | 69 KB | Tensor-to-shard index |
| tokenizer.json | 20.0 MB | Qwen3.5 tokenizer |
| tokenizer_config.json | 1.2 KB | Tokenizer metadata |
| chat_template.jinja | 8.0 KB | Upstream chat template |
| preprocessor_config.json | 390 B | Image preprocessing configuration |
| video_preprocessor_config.json | 385 B | Video preprocessing configuration |
| generation_config.json | 163 B | Generation token configuration |

## Function Calling

The upstream Qwythos model uses the Qwen3.5 chat template and is intended for
tool-augmented and agentic workflows. When a runtime accepts tool definitions,
pass them through the tokenizer or processor chat-template interface and parse
the emitted tool-call blocks in the application.

Tool execution, argument validation, permissions, and safety policy must be
enforced outside the model.

## Limitations and Safety

- This is a format conversion, not a new training run. It does not improve or
  independently validate the upstream model's capabilities.
- The benchmark values above are upstream-reported internal-harness results,
  not measurements reproduced by this conversion.
- The model may hallucinate facts, citations, identifiers, code behavior, or
  tool arguments. Use retrieval, execution checks, and human review when
  correctness matters.
- The upstream release is intentionally less refusal-oriented and may engage
  with sensitive medical, biological, chemical, or security topics. Add
  application-level safeguards and comply with applicable law.
- Do not use the model as the sole authority for medical, legal, financial,
  security-critical, or other high-stakes decisions.
- The configured one-million-token context can require substantial unified
  memory and has not been validated for every modality or deployment.
- The upstream card reports HumanEval below the raw Qwen3.5-9B base and notes
  that MTP acceptance may be modest.
- Visual-language quality was not independently benchmarked as part of this
  conversion.

## License

This MLX conversion is released under the same Apache-2.0 license as the
upstream model.

## Acknowledgements

- Original model:
  [empero-ai/Qwythos-9B-v2](https://huggingface.co/empero-ai/Qwythos-9B-v2)
- Original developer: Empero AI
- Base family: Qwen3.5-9B by Alibaba/Qwen
- MLX ecosystem: Apple MLX and MLX-VLM
- MLX conversion repository:
  xunkutech-ai/Qwythos-9B-v2-MLX-bf16