Image-Text-to-Text
MLX
Safetensors
qwen3_5
apple-silicon
quantized
8-bit precision
multimodal
vision-language
hybrid-attention
speculative-decoding
conversational
4-bit precision
Instructions to use avlp12/Qwen3.8-27B-Alis-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use avlp12/Qwen3.8-27B-Alis-MLX-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("avlp12/Qwen3.8-27B-Alis-MLX-8bit") config = load_config("avlp12/Qwen3.8-27B-Alis-MLX-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use avlp12/Qwen3.8-27B-Alis-MLX-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Qwen3.8-27B-Alis-MLX-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "avlp12/Qwen3.8-27B-Alis-MLX-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use avlp12/Qwen3.8-27B-Alis-MLX-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Qwen3.8-27B-Alis-MLX-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default avlp12/Qwen3.8-27B-Alis-MLX-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use avlp12/Qwen3.8-27B-Alis-MLX-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "avlp12/Qwen3.8-27B-Alis-MLX-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "avlp12/Qwen3.8-27B-Alis-MLX-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Correct vision-preservation claim: mlx-vlm-family community builds keep vision (not MTP)
Browse files- README.md +3 -1
- assets/builds.png +2 -2
- assets/builds.svg +1 -1
- assets/make_builds_png.py +1 -1
README.md
CHANGED
|
@@ -36,7 +36,9 @@ Against bf16 this build is 54% of the size at 1.73× the decode speed, and it is
|
|
| 36 |
|
| 37 |
## The vision tower is intact
|
| 38 |
|
| 39 |
-
Qwen3.8-27B is a vision-language checkpoint (`Qwen3_5ForConditionalGeneration`, `model_type: qwen3_5`), and
|
|
|
|
|
|
|
| 40 |
|
| 41 |
This repo keeps all **333** `model.visual.*` tensors — 0.461B parameters, 0.92 GB — as the **original bf16 bytes**. The tower is not quantized at any bit-width in this set. The shipped `config.json` keeps `vision_config`, declares `Qwen3_5ForConditionalGeneration`, and sets `language_model_only: false`; `preprocessor_config.json` and `video_preprocessor_config.json` ship alongside weights that actually exist.
|
| 42 |
|
|
|
|
| 36 |
|
| 37 |
## The vision tower is intact
|
| 38 |
|
| 39 |
+
Qwen3.8-27B is a vision-language checkpoint (`Qwen3_5ForConditionalGeneration`, `model_type: qwen3_5`), and the standard text-only MLX conversion path **drops the vision weights entirely** — a text-only build carries zero vision tensors and cannot process an image.
|
| 40 |
+
|
| 41 |
+
**Correction (2026-08-16).** An earlier revision of this card claimed this was the first MLX build to keep the vision weights, and that the 12 other public MLX builds then surveyed all carried zero vision tensors. Both claims were wrong when published — the launch-day survey behind them was too narrow. mlx-vlm-family conversions created the same day, `mlx-community/Qwen3.8-27B-{4bit,8bit,mxfp4,nvfp4}` among them, preserve all 333 vision tensors (none of the four carries a single MTP tensor), and a fuller census on 2026-08-16 — 283 MLX-tagged Hub repos matching this model's name, derivatives and sidecar repos included — found text-only builds with zero vision tensors, mlx-vlm-family builds with vision but no MTP head, and other builds carrying **both** subsystems, several published before this set. This card therefore claims no first and no only. What it stands behind is what the weight index proves about these weights — the complete vision tower, unquantized, and the complete MTP head, together — plus the checks below that both paths actually run here: mlx-vlm image input and MTP and DSpark speculative decoding on the same weights. Whether other both-preserving builds load the same way, I have not tested.
|
| 42 |
|
| 43 |
This repo keeps all **333** `model.visual.*` tensors — 0.461B parameters, 0.92 GB — as the **original bf16 bytes**. The tower is not quantized at any bit-width in this set. The shipped `config.json` keeps `vision_config`, declares `Qwen3_5ForConditionalGeneration`, and sets `language_model_only: false`; `preprocessor_config.json` and `video_preprocessor_config.json` ship alongside weights that actually exist.
|
| 44 |
|
assets/builds.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/builds.svg
CHANGED
|
|
|
|
assets/make_builds_png.py
CHANGED
|
@@ -184,7 +184,7 @@ txt(858, 199, "quantizing buys decode, not prefill.", 13, MUT)
|
|
| 184 |
txt(858, 262, "Vision tower kept, unquantized", 15, INK, "bold")
|
| 185 |
txt(858, 284, "All three builds carry 333 vision tensors", 13, MUT)
|
| 186 |
txt(858, 302, "in bf16 (2211 total, MTP head included).", 13, MUT)
|
| 187 |
-
txt(858, 320, "
|
| 188 |
|
| 189 |
txt(858, 380, "Only 16 of 64 layers hold KV", 15, INK, "bold")
|
| 190 |
txt(858, 402, "48 GatedDeltaNet + 16 full attention.", 13, MUT)
|
|
|
|
| 184 |
txt(858, 262, "Vision tower kept, unquantized", 15, INK, "bold")
|
| 185 |
txt(858, 284, "All three builds carry 333 vision tensors", 13, MUT)
|
| 186 |
txt(858, 302, "in bf16 (2211 total, MTP head included).", 13, MUT)
|
| 187 |
+
txt(858, 320, "Text-only MLX conversions carry 0.", 13, MUT)
|
| 188 |
|
| 189 |
txt(858, 380, "Only 16 of 64 layers hold KV", 15, INK, "bold")
|
| 190 |
txt(858, 402, "48 GatedDeltaNet + 16 full attention.", 13, MUT)
|