sahilchachra/Tessera-4B-Preview-optiq-5bpw-mlx

tessera-4b-optiq-5bpw-mlx

MLX quantization of sahilchachra/Tessera-4B-Preview for Apple Silicon.

Variant: OptiQ mixed-precision (target 5.0 bpw)
Disk size: 3094 MB
Quantized by: sahilchachra

About this quantization

Unlike uniform 4-bit quantization (which forces every layer onto the same bit grid and often collapses reasoning at low bit widths), this model was quantized with mlx-optiq using per-layer KL-sensitivity analysis:

1. A small calibration set (32 samples spanning prose, multi-step reasoning, code, and constraint-following instructions) is run through the FP16 reference and through trial quantizations of each layer.
2. The output drift per layer is measured. Layers whose outputs are most affected by quantization (typically the final attention projections, the lm_head, and a few middle blocks) get more bits; layers that tolerate aggressive quantization get fewer.
3. The final assignment hits the target average bits-per-weight while keeping the bits where they matter. This trades off precision unequally so the average comes out near the target (5.0 bits/weight), but the bits that matter most for output fidelity stay high.

Quantization config

• Method: optiq_mixed_precision (mlx-optiq)
• Target bits/weight: 5.0
• Achieved bits/weight: 5.013
• Candidate bits: [4, 6, 8]
• Group size: 64
• Sensitivity reference: uniform_4bit
• Calibration: 32-sample 4-domain mix (prose + reasoning + code + constraints)

Per-layer bit allocation

248 quantizable components total. OptiQ allocated bits non-uniformly based on KL sensitivity:

Bits	Components	Share
8-bit	50	20.2%
6-bit	113	45.6%
4-bit	85	34.3%
Total	248	100.0%

Benchmark results

Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

	This model	Naive 8-bit	FP16 baseline
Decode tok/s (avg, long traces)	76.51	57.3	31.96
Peak memory (GB)	3.776	5.017	8.82
Disk size (MB)	3094	4282	8907

Quality

Benchmark	This model	Naive 8-bit	FP16 baseline	n
MATH-500 (math reasoning)	83.3% (answered 29/30)	86.7% (answered 30/30)	86.7% (answered 28/30)	30
IFEval (instruction following)	33.3%	30.0%	30.0%	30
GSM8K (math, accuracy)	96.7%	96.7%	93.3%	30
HumanEval (code, pass@1)	90.0%	90.0%	86.7%	30
MMLU (knowledge, accuracy)	63.3%	63.3%	56.7%	30

MATH-500 per-level accuracy

Level	This model	Naive 8-bit	FP16 baseline
level 1	83.3%	83.3%	83.3%
level 2	100.0%	100.0%	100.0%
level 3	66.7%	66.7%	66.7%
level 4	83.3%	83.3%	83.3%
level 5	83.3%	100.0%	100.0%

Context scaling (decode tok/s)

Context length	Decode tok/s
~128 tokens	77.9
~256 tokens	77.7
~512 tokens	77.8
~1024 tokens	77.6

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/tessera-4b-optiq-5bpw-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)

All variants in this collection

Model	Variant
sahilchachra/tessera-4b-4bit-mlx	Affine int4
sahilchachra/tessera-4b-8bit-mlx	Affine int8
sahilchachra/tessera-4b-mxfp4-mlx	Block float MX FP4
sahilchachra/tessera-4b-mxfp8-mlx	Block float MX FP8
sahilchachra/tessera-4b-optiq-5bpw-mlx	OptiQ mixed-precision (target 5.0 bpw) ← this model

Notes

• Requires Apple Silicon (M1 or later) with MLX
• Benchmarks run on Apple M5 Pro, 24 GB unified memory
• License: see sahilchachra/Tessera-4B-Preview for the original model's license

Original model

See sahilchachra/Tessera-4B-Preview for full model details and intended use.