can't load model in lm-studio.ai MLX latest version
#1
by Slach - opened
🥲 Failed to load the model
Failed to load model.
Error when loading model: KeyError: 'partial_rotary_factor'
MLX version info:
- mlx-engine==9445b31
- mlx==0.31.2
- mlx-lm==0.31.3
- mlx-vlm==0.6.1
Fixed — the model was tagged as a vision-language model (Qwen3_5ForConditionalGeneration), which made LM Studio try to load it through its VLM engine and fail on partial_rotary_factor. It's actually a text-only model, so I've re-tagged it as a causal LM. Please re-download the model (or delete and re-pull it in LM Studio so it grabs the updated config) and it should load normally. If it still misbehaves, make sure your LM Studio MLX runtime is updated.
remove model and download fresh snapshot
still the same error
🥲 Failed to load the model
Failed to load model.
Error when loading model: KeyError: 'partial_rotary_factor'
Вid you try your model with lm-studio?
❯ ls -la ~/.lmstudio/models/sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx
total 18100392
drwxr-xr-x@ 10 slach staff 320 Jun 24 18:27 .
drwxr-xr-x@ 3 slach staff 96 Jun 24 17:56 ..
-rw-r--r--@ 1 slach staff 7756 Jun 24 17:56 chat_template.jinja
-rw-r--r--@ 1 slach staff 2533 Jun 24 17:56 config.json
-rw-r--r--@ 1 slach staff 164 Jun 24 17:56 generation_config.json
-rw-r--r--@ 1 slach staff 5353758105 Jun 24 18:27 model-00001-of-00002.safetensors
-rw-r--r--@ 1 slach staff 3880967869 Jun 24 18:23 model-00002-of-00002.safetensors
-rw-r--r--@ 1 slach staff 69444 Jun 24 17:56 model.safetensors.index.json
-rw-r--r--@ 1 slach staff 1268 Jun 24 17:57 tokenizer_config.json
-rw-r--r--@ 1 slach staff 19989343 Jun 24 17:57 tokenizer.json
cat ~/.lmstudio/models/sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx/config.json
{
"architectures": [
"Qwen3_5ForCausalLM"
],
"dtype": "bfloat16",
"eos_token_id": [
248046,
248044
],
"model_type": "qwen3_5",
"pad_token_id": 248044,
"quantization": {
"group_size": 32,
"bits": 8,
"mode": "mxfp8"
},
"quantization_config": {
"group_size": 32,
"bits": 8,
"mode": "mxfp8"
},
"text_config": {
"attention_bias": false,
"attention_dropout": 0.0,
"attn_output_gate": true,
"bos_token_id": null,
"dtype": "bfloat16",
"eos_token_id": 248044,
"full_attention_interval": 4,
"head_dim": 256,
"hidden_act": "silu",
"hidden_size": 4096,
"initializer_range": 0.02,
"intermediate_size": 12288,
"layer_types": [
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention"
],
"linear_conv_kernel_dim": 4,
"linear_key_head_dim": 128,
"linear_num_key_heads": 16,
"linear_num_value_heads": 32,
"linear_value_head_dim": 128,
"mamba_ssm_dtype": "float32",
"max_position_embeddings": 1048576,
"mlp_only_layers": [],
"model_type": "qwen3_5_text",
"mtp_num_hidden_layers": 1,
"mtp_use_dedicated_embeddings": false,
"num_attention_heads": 16,
"num_hidden_layers": 32,
"num_key_value_heads": 4,
"pad_token_id": null,
"partial_rotary_factor": 0.25,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"factor": 4.0,
"original_max_position_embeddings": 262144,
"mrope_interleaved": true,
"mrope_section": [
11,
11,
10
],
"rope_theta": 10000000,
"type": "yarn"
},
"tie_word_embeddings": false,
"use_cache": true,
"vocab_size": 248320
},
"tie_word_embeddings": false,
"transformers_version": "5.12.1",
"use_cache": false
}
I tried a different LM Studio version and it had worked. I retried again with the latest one and it failed.
Have pushed a fix in the config and tested with the latest version of LMStudio.
Its working now. Can you try once?