Instructions to use sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Docs: correct LM Studio load-fix note (rope_parameters.partial_rotary_factor)
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ MLX quantization of [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingfac
|
|
| 13 |
|
| 14 |
> **Note — text tower only.** The source model is a **Qwen3.5-VL multimodal** model (`Qwen3_5ForConditionalGeneration`, with a vision encoder). This MLX conversion contains **only the text/language tower** — the vision encoder weights are **not** included, so this is a **text-only** model and does not accept image or video input. The text reasoning the original is benchmarked for (GSM8K, MMLU) is unaffected.
|
| 15 |
>
|
| 16 |
-
>
|
| 17 |
|
| 18 |
**Variant**: Block float MX FP8
|
| 19 |
**Disk size**: 8826 MB
|
|
|
|
| 13 |
|
| 14 |
> **Note — text tower only.** The source model is a **Qwen3.5-VL multimodal** model (`Qwen3_5ForConditionalGeneration`, with a vision encoder). This MLX conversion contains **only the text/language tower** — the vision encoder weights are **not** included, so this is a **text-only** model and does not accept image or video input. The text reasoning the original is benchmarked for (GSM8K, MMLU) is unaffected.
|
| 15 |
>
|
| 16 |
+
> It loads via the standard MLX **LLM** path (mlx-lm, LM Studio). For LM Studio compatibility the config carries `partial_rotary_factor` **inside** `rope_parameters` (LM Studio's engine hard-indexes that key, unlike mlx-lm which defaults it); the config is also tagged as a causal LM (`architectures: ["Qwen3_5ForCausalLM"]`, vision/image/video token ids removed) to reflect that it is text-only.
|
| 17 |
|
| 18 |
**Variant**: Block float MX FP8
|
| 19 |
**Disk size**: 8826 MB
|