Instructions to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx")
model = AutoModelForMultimodalLM.from_pretrained("nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx")
config = load_config("nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx

SGLang

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Unsloth Desktop

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with Docker Model Runner:
```
docker model run hf.co/nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx
```

Hermes Agent

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

can you release Qwen3.6-27B version of this?

by cnsiva - opened Jun 13

Discussion

cnsiva

Jun 13

•

edited Jun 13

Thank you for this model. I’ve finally settled with it for my daily coding tasks. It works great with OMLx. It also performs well, with a speed of approximately 60 in M1 Max and around 100 t/s in M5 Max.

I am curious to test and observe how the Qwen3.6-27B model performs with this.

I’m not sure if this can be ported to Gemma-4-26B-A4B-QAT-it. If it’s possible, that would be great.

Thanks,
Siva

nightmedia

Owner Jun 13

•

edited Jun 13

The closest Claude-Qwen I got is this merge:

Qwen/Qwen3.6-27B
DavidAU/Qwen3.5-27B-Claude-4.6-OS-INSTRUCT
DavidAU/Qwen3.6-27B-Heretic2-Uncensored-Finetune-Thinking

https://huggingface.co/nightmedia/Qwen3.6-27B-Architect-Polaris-mxfp8-mlx

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.709,0.874,0.909,0.788,0.494,0.820,0.768
mxfp4    0.698,0.874,0.909,0.786,0.494,0.820,0.759

Quant    Perplexity      Peak Memory   Tokens/sec
mxfp8    3.923 ± 0.025   34.74 GB      193

It is also the highest IQ ever reached in a 27B, as far as I know.

I will see if I can add MTP layers to it :)

cnsiva

Jun 14

Thank you.
how is this differ from https://huggingface.co/nightmedia/Qwen3.6-27B-Architect-Polaris-Deckard-qx86-hi-mlx ?

nightmedia

Owner Jun 14

•

edited Jun 14

The Deckard has PKD in it, and the behavior is quite different.

I like PKD when I explore the personalities interacting in the Holodeck, because it aims to separate the voice to be more "in character" when so claimed. The non-PKD is a bit more formal about it. The price you pay is slightly lower arc_challenge, but that's really high as it is, beyond what the model was designed to deliver.

The irony is, this is a better Claude than the current Claude, since they dumbed it down ;)

cnsiva

Jun 16

Is there an update on this model for the MTP? The name has MTP, but seeing this in oMlx

nightmedia

Owner Jun 16

•

edited Jun 16

I am working now on a fix for this.

The MTP layers were dropped in the merge, and I missed that.

The idea is to provide an alternative config and the missing tensors, just have to wait for the current test queue to finish :)

nightmedia

Owner Jun 16

I set up two experiments here, configuring MTP in the two generally accepted formats: inline and sidecar.

Each works in its own setting, so I included the config folders to be easily swapped and experimented with.

https://huggingface.co/nightmedia/Qwen3.6-28B-PKD-Darwin-qx86-hi-mlx

If either of these proves to work, or there are simple fixes that bring it to work, I can apply the same method to the MoE

The tensors from the 27B are not compatible with the 35B, but are easily extracted and quanted with a python script

cnsiva

Jun 16

Thank you very much. I’ll test it and let you know.

cnsiva

Jun 16

I like your qx64 version than qx86 version 😀 I am using M1 Max with 64 GB. The qx64 works better. I will try out this version and update you.

nightmedia

Owner Jun 16

•

edited Jun 16

yeah there is that thing about the qx64-hi, it is smarter. It does show in numbers too, and that's quite silly to think that a 6/4 bit mix could match bf16, but hey, performance under constraints does wonders to morale :)

I am publishing the qx86-hi first, don't want people to feel short-changed, but in this case, you are right, smaller is better in this case.

cnsiva

Jun 17

oMlx doesn’t detect the MTP. I have also tried copying the MTP files to the model root directory, but it still doesn’t detect them.

nightmedia

Owner Jun 17

thanks for trying, I will find some time tomorrow to install omlx and figure out what happens. Can you point me to a model that you know it works with MTP on omlx so I can get some idea what is going on?

cnsiva

Jun 17

Here’s the model that works for me.
https://huggingface.co/dealignai/Qwen3.6-27B-MXFP4-CRACK-MTP

Alternatively, if I perform the OQ Quantization from the OMLX, it works for me from the OMLX.

nightmedia

Owner Jun 17

I think I know what happens, the tensors from the side car are prefixed with mtp, the others have language_model.mtp. It all depends how the parser interprets the config file and the indexes. I will try changing those tomorrow.

cnsiva

Jun 17

I just have tested this model against the llama.cpp repository and done some analysis. While the model is indeed impressive, it is quite slow and only achieves a throughput of 10 t/s on my M1 Max. In contrast, the “nightmedia/Qwen3.6-35B-A3B-MTP-Holo3-Qwopus-Coder-qx64y-hi-mlx” model is remarkably 5 times faster. It’s possible that the qx64y-hi version of this with MTP may offer improved performance for my specific setup.

nightmedia

Owner Jun 18

I have a new version

https://huggingface.co/nightmedia/Qwen3.6-35B-A3B-Qwable-Holo3-Qwopus

This is a merge with Qwable, but it's raw source, not BF16. I noticed the MTP tensors get dropped when converting to BF16, which was the reason they were missing in the first place from the previous models. You can use your preferred conversion method to extract the correct format, until I figure out a way to streamline it.

The qx64-hi is the same formula as the qx64y-hi, as that was an evolutionary step in the Deckard(qx) formula to better support the specifics of the new model.

-G

nightmedia

Owner Jun 18

There's also an 27B-Architect-Polaris-Fable in the works :)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment