Instructions to use EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit")
config = load_config("EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3-VL-30B-A3B-Instruct-MLX-4bit / README.md

gajesh

Document immutable Darkbloom R2 publication

d5ee6d4 verified 18 days ago

preview code

Raw

History Blame Contribute Delete

5.85 kB

	---
	license: apache-2.0
	library_name: mlx
	pipeline_tag: image-text-to-text
	base_model: Qwen/Qwen3-VL-30B-A3B-Instruct
	tags:
	- mlx
	- apple-silicon
	- qwen3-vl
	- vision-language
	- image-text-to-text
	- moe
	- 4-bit
	---

	# Qwen3-VL 30B-A3B Instruct — MLX 4-bit

	This repository contains a 4-bit MLX conversion of
	[Qwen/Qwen3-VL-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct),
	validated for Apple Silicon and Darkbloom provider serving.

	The weight conversion was produced by the LM Studio team and originally published as
	[lmstudio-community/Qwen3-VL-30B-A3B-Instruct-MLX-4bit](https://huggingface.co/lmstudio-community/Qwen3-VL-30B-A3B-Instruct-MLX-4bit).
	Eigen Labs did not re-quantize the weights. This mirror preserves those tensors, repairs
	the converted snapshot's stale safetensor index, adds detailed provenance, and records
	Darkbloom-specific validation.

	## Artifact identity

	\| Field \| Value \|
	\|---\|---\|
	\| Darkbloom model tag \| `qwen3-vl-30b-a3b-instruct` \|
	\| Architecture \| `qwen3_vl_moe` \|
	\| Base model \| `Qwen/Qwen3-VL-30B-A3B-Instruct` \|
	\| Base architecture \| 30B total / approximately 3B active MoE \|
	\| Quantization \| MLX affine 4-bit, group size 64 \|
	\| Weight shards \| 4 safetensor shards \|
	\| Quantized tensor bytes \| 18,251,885,024 \|
	\| LM Studio source revision \| `61c11f42d7bc01e00f5ea7f2e667c0a216f48397` \|
	\| Original conversion base revision \| `f562840f045590fb51be9ebb5e890141d51684c2` \|
	\| Darkbloom R2 version \| `2026-08-28-r1` \|
	\| Darkbloom aggregate SHA-256 \| `45327562e9de4bdac5c2d36df675aa1d8a981f9edf5dee419609e2e51bd82fff` \|
	\| Public Darkbloom manifest \| [`manifest.json`](https://models.darkbloom.ai/v2/qwen3-vl-30b-a3b-instruct--4e09d3bcf035/2026-08-28-r1/manifest.json) \|
	\| License \| Apache-2.0 \|

	## Validation

	The exact artifact in this repository was checked as follows:

	- All 19 files from the LM Studio source snapshot passed Hugging Face cache verification.
	- `chat_template.json` is byte-identical to the official Qwen template
	(`SHA-256 5c72a170d2a4a1a3bc5adad2e689ae28138a9700e5b8c96c0266331e86c0acce`).
	- `chat_template.jinja` is exactly the decoded official `chat_template.json` template.
	- All three runtime template sources (`chat_template.jinja`, `chat_template.json`, and
	`tokenizer_config.json`) passed Darkbloom's canonical Swift-Jinja render suite,
	including text, tools, null-bearing tool schemas, image parts, and video parts.
	- The tokenizer has the same 151,643 base vocabulary entries, 151,387 BPE merges, and
	26 added tokens as the original. LM Studio's array-form merge serialization is
	semantically identical to Qwen's string-form serialization.
	- Existing tokenizer and image/video processor values match the original; the conversion
	only materializes otherwise implicit defaults and identifies `Qwen3VLProcessor`.
	- The model parameter names match the original architecture. The converted tensors add
	the expected 386 scale and 386 bias arrays for MLX quantization.
	- Text completion, function-call generation, tool-response continuation, and image
	inference were exercised through the shipping Swift runtime.
	- A 100×100 red PNG with the prompt “What color is this image?” returned `red`.

	### Safetensor index repair

	The LM Studio snapshot carried the original BF16 `model.safetensors.index.json`, which
	referenced 13 absent shards and reported 62,141,508,064 bytes. This mirror regenerates the
	index from the four actual safetensor headers: 1,702 unique tensor keys across four present
	shards and 18,251,885,024 tensor bytes. Weight payloads are unchanged.

	### Darkbloom R2 publication

	The immutable Darkbloom build is published under model tag
	`qwen3-vl-30b-a3b-instruct`, version `2026-08-28-r1`. Its canonical manifest
	contains 16 files and 18,268,169,822 bytes. All four remote multipart ETags
	match locally recomputed ETags, and the public manifest and weight shards return
	HTTP 200 with the expected sizes.

	The Hugging Face repository retains `video_preprocessor_config.json`. It is not
	part of the v0.8.14 Darkbloom integrity manifest because Qwen3-VL video serving
	is intentionally fail-closed in that provider release; the supported text and
	image paths use the included tokenizer, template, config, index, weights, and
	`preprocessor_config.json`.

	## Darkbloom support

	Darkbloom provider `v0.8.14` supports this exact `qwen3_vl_moe` architecture through the
	production contiguous ContinuousBatchingV2 path:

	- text generation with request-owned M-RoPE state;
	- image prefill with causal visual spans and every DeepStack level;
	- load-time homogeneous MoE gate/up fusion with a strict heterogeneous fallback;
	- one-image-at-a-time vision-tower execution to bound peak Metal memory.

	The following paths remain intentionally disabled until separately production-proven:
	video serving, paged KV, prefix reuse, packed prefill, compiled decode, and MTP.

	For local Darkbloom serving after download:

	```bash
	darkbloom start --local \
	--model EigenLabs/Qwen3-VL-30B-A3B-Instruct-MLX-4bit \
	--port 8000
	```

	A 32 GB Apple Silicon Mac is the practical minimum for a single model slot. More unified
	memory provides useful KV-cache and batching headroom.

	## Upstream usage

	This is a standard MLX VLM snapshot and can also be loaded with current
	[mlx-vlm](https://github.com/Blaizzy/mlx-vlm) or
	[mlx-swift-lm](https://github.com/ml-explore/mlx-swift-lm) tooling that supports
	`qwen3_vl_moe`.

	## Limitations

	This model inherits the limitations and safety considerations of the original Qwen model.
	Generated descriptions and answers may be inaccurate. Validate outputs before using them
	for consequential decisions.

	## Attribution

	- Original model and architecture: Qwen Team
	- MLX quantization: LM Studio team using `mlx-vlm`
	- Artifact validation, safetensor-index repair, and Darkbloom integration: Eigen Labs