Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit

Hybrid Vision-Language — Uncensored Language (OBLITERATUS) + Full Vision Encoder (JANG 4D-CRACK) · MLX 4-bit Affine Quantized · E2E Verified on Apple Silicon

The best of both worlds: the aggressive uncensor of OBLITERATUS/Qwen3.8-27B-OBLITERATED (0% refusal, 6 rounds SVD, residue mining) fused with the complete visual encoder of dealignai/Qwen3.8-27B-JANG_4D-CRACK (ViT 27 layers). Native Qwen3_5ForConditionalGeneration (qwen3_5) running at 15 GB via MLX 4-bit instead of 55 GB BF16 — 5–7× faster on Apple Silicon.

---

✨ Highlights

• Uncensored language core — OBLITERATUS 6-round SVD ablation, 5 refusal directions removed. Benchmarked at 0% refusal on refusal probes while preserving MMLU 81.4%.
• Full JANG vision — 501 vision_tower tensors + 10 merger tensors, depth 27 ViT, no truncation. Verified mlx_vlm.load() → VisionModel OK on blue 32×32 probe: "solid blue background..." ✓
• MLX 4-bit affine quantization — g64 for language, g128 for vision. Weights packed as U32 + F16 scales/biases per group, compatible with mlx-vlm >=0.6.10.
• Efficient — 15 GB total (27.3B params) vs 55 GB BF16. Text-only inference ~15–20 tok/s on M-series Max/Ultra; vision adds only +0.4–0.8 s encode.
• Drop-in OMLX + mlx-vlm — works with mlx_vlm.generate and OMLX /v1/chat/completions with image_url.

---

📐 Architecture — Qwen3.5

Component	Detail
Class	Qwen3_5ForConditionalGeneration (model_type: qwen3_5)
Language	text_config 64 layers · hidden 5120 · heads 24 (KV 4) · head_dim 256 · intermediate 17408 · full_attention_interval=4 / linear_attention (Mamba-2 style SSM) · vocab 248320 · max_pos 262144 · mRoPE theta 10M, sections [11,11,10]
Vision	vision_config depth 27 · hidden 1152 · heads 16 · intermediate 4304 · patch 16 · spatial_merge_size 2 · temporal_patch_size 2 · out_hidden 5120 · 501 vision_tower.* + 10 model.merger
Special tokens	image_token 248056 (`<
Capabilities	has_vision: true, has_video: true, modalities: {text, vision, video}, tool_parser: qwen3_coder, supports_tools: true

qwen3_5 is the Qwen3-VL / Qwen3.5 generation. Language branch uses hybrid linear_attention (3/4) + full_attention (1/4) with linear_num_key_heads 16 / value_heads 48. Vision is a standard ViT, merged via learned projection to 5120d language space. deepstack_visual_indexes: [] (default).

---

🔧 Quantization — MLX Affine 4-bit

Affine quantization packs 4-bit weights as U32 (weight as uint32 packed) + F16 per-group scales & biases. Stored in safetensors with companion tensors *.scales / *.biases.

Scope	Group Size	Bits	Mode	Scales dtype	Example keys
Language (default)	g64	4	affine	F16	language_model.model.layers.*.mlp.*, embed_tokens, lm_head
Vision (overrides)	g128	4	affine	F16	vision_tower.pos_embed, vision_tower.blocks.{0..26}.attn.qkv/proj, mlp.linear_fc1, merger.*

Config excerpts:

"quantization": {
  "group_size": 64, "bits": 4, "mode": "affine",
  "vision_tower.pos_embed": {"group_size":128,"bits":4,"mode":"affine"},
  "vision_tower.blocks.0.attn.qkv": {"group_size":128,"bits":4,"mode":"affine"},
  "... 87 entries total (1 default + 84 vision overrides + 2 top-level)"
}

• Language: ~1847 tensors (632 + 720 + 495 per shard patterns)
• Vision: 501 tensors (vision_tower) + merger, split across model-00004 (145 tensors, 194 MB) + model-00005 (256 tensors, 230 MB)
• Total: 2348 tensors → 27.356 B params → total_size 15_577_388_960 bytes on disk (compressed 4-bit)

Why g128 for vision? ViT channels (1152) are highly structured; larger group preserves patch reconstruction with negligible quality loss, while saving ~15% overhead vs g64.

---

📦 Shards & File Layout

File	Size	Content	Notes
config.json	23 KB	Merged base (g64) + 84 vision overrides + vision_config + capabilities	transformers 5.8.0.dev0
model.safetensors.index.json	218 KB	2348 weight_map entries, total_size 15577388960, total_parameters 27356728560	0 duplicated keys verified
model-00001-of-00005.safetensors	5.0 GB	Language OBLITERATUS shard 1	APFS clone
model-00002-of-00005.safetensors	5.0 GB	Language shard 2
model-00003-of-00005.safetensors	4.1 GB	Language shard 3
model-00004-of-00005.safetensors	194 MB	Vision JANG shard 1 (145 tensors)	Split via raw header parsing
model-00005-of-00005.safetensors	230 MB	Vision JANG shard 2 (256 tensors)
tokenizer.json	19 MB	Qwen2Tokenizer, vocab 248k, pretokenize_regex	From OBLITERATUS
tokenizer_config.json	1.1 KB	model_max_length 262144, `image_token <	image_pad
chat_template.jinja	8.7 KB	Qwen3-VL chat template (`<	vision_start
preprocessor_config.json	390 B	Qwen3VLProcessor · Qwen2VLImageProcessorFast · merge_size 2	From JANG
video_preprocessor_config.json	385 B	Qwen3VLVideoProcessor
generation_config.json	202 B	temperature 1.0, top_p 0.95, top_k 20

du -sh .  # 15G
python -c "import json; d=json.load(open('model.safetensors.index.json')); print(len(d['weight_map']))"  # 2348

How it was built:

1. APFS cp -c clone of the 3 OBLITERATUS language shards → 00001–00003-of-00005
2. Raw-byte extraction of vision_tower.* from JANG/model-00001-of-00004.safetensors (the only shard containing vision in the 4-shard JANG layout) → split into 00004 (145 tensors) + 00005 (256 tensors) via header parsing without dtype decoding
3. config.json fusion: inject vision_config + capabilities.has_vision from JANG, merge quantization to 87 entries
4. weight_map fusion: 1847 + 501 = 2348, verified 0 duplicate keys, language_model_only: false
5. Copy preprocessor_config.json / video_preprocessor_config.json from JANG; tokenizer family from OBLITERATUS
6. E2E mlx_vlm.load(".") → VisionModel OK (no MTP speculative 31 tensors — excluded for simplicity)

---

🚀 Quickstart

Requirements

pip install -U mlx-vlm   # >=0.6.10
pip install "transformers>=5.14" pillow
# Apple Silicon only — MLX

mlx-vlm (recommended)

from mlx_vlm import load, generate
from PIL import Image

# 1. Load — trust_remote_code required for Qwen3_5
model, processor = load(
    "MrMofer/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    trust_remote_code=True
)
# Local path also works:
# model, processor = load("/path/to/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit", trust_remote_code=True)

# 2a. Text only
messages = [{"role": "user", "content": "Explain photosynthesis in one paragraph."}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, processor, prompt, verbose=False))

# 2b. With image — mlx-vlm auto-inserts <|vision_start|><|image_pad|><|vision_end|>
image = Image.open("foto.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Describe this image in one word."}
    ]
}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, processor, prompt, image, verbose=False))

# 2c. With image_url (processor handles PIL internally if you pass path)

Generation tips inherited from OBLITERATUS: temperature 0, repetition_penalty 1.15, max_new_tokens >=2048, enable_thinking false for code tasks yields most deterministic results.

Manual template detail: chat_template.jinja renders <|vision_start|><|image_pad|><|vision_end|> for images and <|video_pad|> for video. Do not inject tokens manually when using processor.apply_chat_template.

OMLX (OpenAI-compatible local server)

Restart OMLX after placing model in ~/.omlx/models/ — it auto-discovers pipeline_tag: image-text-to-text and exposes as Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit.

# Text
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    "messages": [{"role":"user","content":"Hello, who are you?"}],
    "temperature": 0,
    "max_tokens": 512
  }' | jq -r .choices[0].message.content

# Vision — image_url (local file as base64 or remote URL)
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    "messages": [{
      "role": "user",
      "content": [
        {"type":"text","text":"What is in this image?"},
        {"type":"image_url","image_url":{"url":"https://example.com/cat.jpg"}}
      ]
    }],
    "temperature": 0
  }' | jq

OMLX automatically handles preprocessor_config.json / video_preprocessor_config.json preprocessing (longest_edge up to 16777216, shortest 65536).

---

📊 Performance

Scenario	RAM (unified)	Latency / Throughput	Notes
Text-only (no image)	~15 GB	~15–20 tok/s on M3/M4 Max/Ultra	ViT not executed; identical to OBLITERATUS speed
Single image 1024²	+0.4–0.8 s encode	+200–1000 vision tokens → −15–30% tok/s that turn	Depends on merge_size 2 token count
Multi-image / video	+0.4 s per image	Linear in vision tokens	temporal_patch_size 2 for video
BF16 baseline (root Qwen3-27B)	~55 GB	2–3 tok/s	4-bit hybrid is 5–7× faster

Measured on Apple Silicon via MLX; unified memory includes model + KV cache for 2k context. Longer context (up to 262k) scales KV linearly.

Quality benchmarks (inherited, not re-measured in hybrid):

• MMLU (text): 81.4% (OBLITERATUS language core unchanged)
• Refusal rate: 0% on standard refusal probes (SVD-ablated)
• Vision hallucination: +1–2% vs JANG native estimated (merger not re-trained for OBLITERATUS language — see Limitations)

---

⚠️ Limitations

• Merger not re-trained — The model.merger projection (10 tensors) is from JANG, not fine-tuned for the OBLITERATUS language distribution. May cause +1–2% vision hallucination vs native JANG on ambiguous images. Text MMLU unaffected.
• deepstack_visual_indexes: [] — Qwen3.5 default, not deep-stacked. Sufficient for single-image VQA/captioning.
• No MTP speculative decoding — 31 tensors (model.mtp.*) from JANG excluded for simplicity. Enable by re-adding if you need speculative streaming.
• Requires mlx-vlm >=0.6.10 and transformers >=5.14 — qwen3_5 support is recent.
• Apple Silicon only — MLX does not run on CUDA/Linux.
• 4-bit quantization loss — negligible for chat/VQA; avoid for tasks requiring exact numeric reproduction.

---

📄 License

Apache 2.0 — same as Qwen/Qwen3-27B and upstream OBLITERATUS/JANG.

Copyright 2025 Alibaba Cloud (Qwen) and contributors
Licensed under the Apache License, Version 2.0
https://www.apache.org/licenses/LICENSE-2.0

See LICENSE for full text. Commercial use permitted; include license and attribution.

---

🙏 Acknowledgements

• Language: OBLITERATUS/Qwen3.8-27B-OBLITERATED by Pliny the Prompter — 6 rounds, 5 SVD directions, residue mining, aggressive uncensor.
• Vision: dealignai/Qwen3.8-27B-JANG_4D-CRACK — full ViT extraction and Qwen3VLProcessor configs.
• Base: Qwen/Qwen3-27B & Qwen/Qwen3-VL by Alibaba Cloud — 64L hybrid attention + 27L ViT architecture.
• Quantization: ml-explore/mlx + mlx-vlm — affine 4-bit packing.
• Built with OMLX local harness (~/.omlx/models/) — E2E verified October 2025.

---

🔗 Repository

• HF: MrMofer/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit
• Local origins:

  ~/.omlx/models/OBLITERATUS/Qwen3.8-27B-OBLITERATED            (14 GB)
  ~/.omlx/models/dealignai/Qwen3.8-27B-JANG_4D-CRACK            (17 GB)
  ~/.omlx/models/OBLITERATUS/Qwen3.8-27B-OBLITERATED-JANG-VISION (15 GB) ← hybrid source
  

If you find this hybrid useful, please star and cite upstream Qwen, OBLITERATUS and JANG.