Ornith-1.5-35B-A3B-abliterated-W8A8

INT8 W8A8 quantisation of huihui-ai's abliterated Ornith 1.5 35B-A3B, with the vision tower, the MoE routers and the MTP head deliberately left in bf16.

35.7 GiB, compressed-tensors 0.18.0, format int-quantized. Architecture Qwen3_5MoeForConditionalGeneration: vision intact, MTP intact.

Lineage

Every link is pinned to the commit this artifact was actually built from.

step	source	revision
foundation	ornith-ai/Ornith-1.5-35B-A3B	upstream release
abliteration (tower)	huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated	7269f0953054d68b3cacbd9ad729921258732007
MTP head	shisa-ai/Ornith-1.5-35B-A3B-MTP-ONLY	2b19b31bfe1659c6b0d9459ec3cbd87e34a322ef
quantisation	llmcompressor QuantizationModifier, recipe in recipe.yaml	this repo

The MTP head is not the one the base model ships, and that is deliberate. It is grafted from shisa-ai's distill. See "The MTP head" below, including a correction to a claim about the original head that circulates and is wrong.

Why this abliteration

Three abliterations of Ornith 1.5 35B-A3B are published. Only huihui-ai's keeps the model whole, and this quantisation preserves that:

	this repo	the other published W8A8	the other abliterations
abliterated	yes	no	yes
architecture	Qwen3_5MoeForConditionalGeneration	Qwen3_5MoeForCausalLM	one downgrades to ForCausalLM
model.visual.* tensors	333	0	0
MTP head	yes (grafted)	yes (original, unfused)	dropped

The two alternatives drop both the MTP head and the vision tower (64.6 GiB against the official 67.0 GiB, zero of each). A text-only checkpoint silently drops every image and OCR caller, which is a failure that looks like nothing at all: the model answers, it just cannot see.

What was quantised, and what was not

Scheme W8A8: 8-bit integer weights, symmetric, per channel, and 8-bit integer activations, dynamic. Targets Linear. No calibration data (data-free round-to-nearest), so nothing about this checkpoint is tuned to a calibration corpus.

Left in bf16, on purpose:

• the whole vision tower, all 27 blocks and the merger
• every MoE router: mlp.gate and shared_expert_gate on all 40 layers. Routers pick experts, and an 8-bit router picks different experts
• the linear_attn modules and their norms. Ornith 1.5 is a hybrid, and the linear-attention state is precision-sensitive
• lm_head
• the entire MTP head (re:.*mtp.*)

The MTP head

The head here is shisa-ai's distill, not the one Ornith ships. Three reasons, and the first thing to say is what is not a reason.

The original head is trained. The widespread claim that it is not is false. The claim (from the model's own discussion #10) is that every projection has a standard deviation of exactly 0.0200, i.e. initializer_range. Probing the published safetensors directly on 2026-08-27 does not support it: mtp.norm.weight has mean +1.9251, mtp.layers.0.post_attention_layernorm.weight +0.8686, and the projection standard deviations vary by a factor of three (mtp.fc 0.0085, shared_expert.gate_proj 0.0089, mlp.gate 0.0096, o_proj 0.0185, q_proj 0.0247). A single init at initializer_range would put all of them at 0.0200 and leave the norms near 0.02. For scale, the same checkpoint's trained model.language_model.norm.weight is 1.6397 against the head's 1.9251. Please do not repeat the claim.

The actual reasons:

1. It accepts nearly twice as many drafted tokens. Measured in vLLM by shisa-ai and published alongside the head: Ornith's own head accepts 37.20% (mean accepted run 2.116), a zero-shot graft of the Qwen3.6 head 50.19% (2.506), and this head, a 12K KL distill over two mixed-data epochs warm-started from Ornith's own, 69.27% (3.078).
2. It transfers. It is a fine-tune rather than a retrain: element-wise against the official head the deltas are 0.5% to 8.7% of each tensor's range. A head only has to be accepted by the tower it drafts for, and this one is within a few percent of the head that already pairs with it.
3. Layout. Ornith ships the head's 256 experts unfused, as 785 separate mtp.layers.0.mlp.experts.N.* tensors. This head ships the 19-tensor fused form (experts.gate_up_proj [256, 1024, 2048], experts.down_proj [256, 2048, 512]).

DFlash was the other candidate and was rejected: its acceptance falls from 89.8% at 8K context to 48.8% near 256K, it is slower than plain autoregressive decode at 252K/8K (0.67x), and it reportedly breaks multimodal.

Why W8A8 rather than FP8 or NVFP4

This matters on Ampere and not much anywhere else. vLLM's W8A8 INT8 scheme declares a minimum compute capability of 75, so an A40 (8.6) multiplies in its own INT8 tensor cores. The FP8 W8A8 scheme declares 89, so below Lovelace vLLM resolves the checkpoint to the weight-only scheme and serves it through Marlin: the activations go back to 16 bits and the memory saving is real while the speedup is not. NVFP4 has no native FP4 path on Ampere at all.

So on Ada, Hopper and Blackwell, prefer the official FP8 build. On Ampere, this one.

Serving

vllm serve nuoram/Ornith-1.5-35B-A3B-abliterated-W8A8 \
  --tensor-parallel-size 2 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml

Fits 2x 48GB (A40, A6000) with room for a long context.

Credit and licence

The model is ornith-ai's, the abliteration is huihui-ai's, and the MTP head is shisa-ai's. This repository changes the numeric format and nothing else. Licence follows the upstream Ornith release; if that differs from the Apache-2.0 declared here, upstream governs.

Abliterated models have had refusal behaviour removed. You are responsible for what you do with it.