Instructions to use avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic")
config = load_config("avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

avlp12 commited on Jul 15

Commit

657cda2

verified ·

1 Parent(s): 9f57d63

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +13 -0

README.md CHANGED Viewed

@@ -28,6 +28,7 @@ Apple Silicon (MLX) **mixed-precision** quantizations of [Qwen/Qwen3.6-35B-A3B](
 |---|---|---|---|---|---|---|
 | **`main`** | **4.000** | **17.33 GB** | 24 GB+ | **102.1 tok/s** | **0.292 (19.7%)** | ✅ golden — GPQA-D **76.3%** |
 | `16gb` | 2.700 | **11.70 GB** | **16 GB** (sysctl) | **110.8 tok/s** | 1.111 (32.9%) | ⚠️ under review — see note |
 | `mtp-bf16` | bf16 | 1.69 GB | — | — | — | MTP head sidecar for self-speculative decoding |
 \* single-stream decode on an M3 Ultra, short prompt. A3B routing makes this the **fastest model class per GB** in the lineup — ~3× the decode rate of the dense 27B sibling.
@@ -45,6 +46,18 @@ Everything below is **measured, not projected** — KL / top-1 flip on a fixed 4
   With int8 KV that leaves ~1.5 GiB of cache ≈ **150 K tokens** of context (KV here is tiny — see below). 111 tok/s decode makes this the best speed-per-GB option for 16 GB machines; prefer it over the dense 27B 16 GB build when throughput matters more than per-token quality.
 ## Usage
 ```python

 |---|---|---|---|---|---|---|
 | **`main`** | **4.000** | **17.33 GB** | 24 GB+ | **102.1 tok/s** | **0.292 (19.7%)** | ✅ golden — GPQA-D **76.3%** |
 | `16gb` | 2.700 | **11.70 GB** | **16 GB** (sysctl) | **110.8 tok/s** | 1.111 (32.9%) | ⚠️ under review — see note |
+| [`vision`](https://huggingface.co/avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic/tree/vision) | 4.087 | **17.93 GB** | 24 GB+ | ~100 tok/s | (same text weights) | ✅ multimodal — vision tower restored at 8-bit |
 | `mtp-bf16` | bf16 | 1.69 GB | — | — | — | MTP head sidecar for self-speculative decoding |
 \* single-stream decode on an M3 Ultra, short prompt. A3B routing makes this the **fastest model class per GB** in the lineup — ~3× the decode rate of the dense 27B sibling.
   With int8 KV that leaves ~1.5 GiB of cache ≈ **150 K tokens** of context (KV here is tiny — see below). 111 tok/s decode makes this the best speed-per-GB option for 16 GB machines; prefer it over the dense 27B 16 GB build when throughput matters more than per-token quality.
+## Multimodal (`vision` branch)
+The base checkpoint ships the full vision tower; `main` (mlx-lm path) is text-only, the
+`vision` branch restores it — identical 4.0bpw text weights + the vision tower at 8-bit.
+Verified with EN/KO OCR tests.
+```bash
+pip install mlx-vlm
+python -m mlx_vlm generate --model avlp12/Qwen3.6-35B-A3B-Alis-MLX-Dynamic --revision vision \
+  --image photo.jpg --prompt "이미지의 텍스트를 읽어줘." --max-tokens 300
+```
 ## Usage
 ```python