---
base_model: zed-industries/zeta-2.1
tags:
- mlx
- optiq
- quantized
- mixed-precision
- 4bit
- 8bit
- apple-silicon
- edit-prediction
- next-edit-suggestion
- code
- autocomplete
- fim
license: apache-2.0
language:
- en
library_name: mlx
pipeline_tag: text-generation
---

# bouroo/zeta-2.1-OptiQ-4

A 4.0-bits-per-weight target, mixed-precision MLX conversion of [zed-industries/zeta-2.1](https://huggingface.co/zed-industries/zeta-2.1), tuned for **code edit-prediction / autocomplete**. Sensitive tensors stay 8-bit; robust tensors drop to 4-bit.

> "4-bit" / "8-bit" here are per-tensor choices from a mixed-precision allocation, not a global bit-width. Achieved BPW is the size-weighted average.

## Variants

Pick the size/quality trade-off:

| Variant | Target BPW | Achieved BPW | 8-bit tensors | 4-bit tensors | Size |
|---|---|---|---|---|---|
| [-OptiQ-4](https://huggingface.co/bouroo/zeta-2.1-OptiQ-4) | 4.0 | 4.692 | 75 | 150 | 5.23 |
| [-OptiQ-5](https://huggingface.co/bouroo/zeta-2.1-OptiQ-5) | 5.0 | 5.279 | 104 | 121 | 5.75 |
| [-OptiQ-6](https://huggingface.co/bouroo/zeta-2.1-OptiQ-6) | 6.0 | 6.145 | 150 | 75 | 6.52 |

## Quantization Details

| Property | Value |
|---|---|
| Method | optiq (calibration-driven KL sensitivity) |
| Target bits per weight | 4.0 |
| Achieved bits per weight | 4.692 |
| Candidate bits | 4, 8 |
| Tensors 8-bit (sensitive) | 75 |
| Tensors 4-bit (robust) | 150 |
| Group size | 64 |
| Reference signal | bf16 |
| Calibration samples | 8 (OptiQ mix) |
| Size on disk | 5.23 GB |

Per-tensor allocation is in `optiq_metadata.json` and `config.json` under `quantization`. A `generation_config.json` ships autocomplete-optimized defaults (`temperature` 0.2, `top_p` 0.95, `max_new_tokens` 128, `eos_token_id` 2).

## About the base model

[Zeta 2.1](https://huggingface.co/zed-industries/zeta-2.1) is a **code edit-prediction model** (next-edit suggestion) finetuned from `ByteDance-Seed/Seed-Coder-8B-Base` — an 8B dense Llama (32 layers, GQA x8 KV heads, 32k context, BF16). Given context + an editable region, it predicts the rewritten region.

## Prompt format (edit-prediction / FIM)

This is a **completion model** (no chat template). It uses Zeta's SPM-style format with markers. Minimal insertion prompt (empty region):

```
<[fim-suffix]>{code after cursor}
<[fim-prefix]><filename>{file_path}
{code before cursor}<|marker_1|><|marker_2|>
<[fim-middle]>
```

For an **edit** (rewrite an existing region), wrap the current region content with the markers and put `<|user_cursor|>` where the cursor lands:

```
<[fim-suffix]>{code after region}
<[fim-prefix]><filename>{path}
{related files / edit_history, each prefixed with <filename>}
<filename>{path}
{before}<|marker_1|>{current part A}<|user_cursor|>{current part B}<|marker_2|>{after}
<[fim-middle]>
```

The model generates `<|marker_1|>{predicted part A}<|user_cursor|>{part B}<|marker_2|>` and **stops at EOS** (`<[end_of_sentence]>`, id 2). Stop on `<|marker_2|>` / EOS.

Minimal `mlx_lm` example:

```python
from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-4")
prompt = "<[fim-suffix]>{suffix}\n<[fim-prefix]><filename>app.py\n{prefix}<|marker_1|><|marker_2|>\n<[fim-middle]>"
out = generate(model, tokenizer, prompt=prompt, max_tokens=128, temp=0.2, top_p=0.95)
# out starts with <|marker_1|>, stop at <|marker_2|>
```

## Usage

### MLX (mlx_lm)

```python
from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-4")
out = generate(model, tokenizer, prompt=fim_prompt, max_tokens=128, temp=0.2, top_p=0.95)
```

### OptiQ serve

```bash
pip install mlx-optiq
optiq serve --model bouroo/zeta-2.1-OptiQ-4
```

### LM Studio

```bash
lms get https://huggingface.co/bouroo/zeta-2.1-OptiQ-4
lms load zeta-2.1-optiq-4   # LM Studio normalizes the key (no namespace); run `lms ls` to confirm
```


## Verification

Confirmed with `mlx_lm` (load + edit-prediction generation) and LM Studio (`lms get` + `lms load`).

---

Quantized and published under Apache-2.0, inherited from the base model.