# Ornith-1.0-35B IQ4_XS Bootstrap MTP Profile

Date: 2026-06-27

Hardware: single RTX PRO 6000 Blackwell Max-Q 96GB, `tp=1`.

Backend: local `llama.cpp` CUDA server from
`vendor/llama.cpp/build-cuda/bin/llama-server`.

## Why This Exists

The released `deepreinforce-ai/Ornith-1.0-35B` config advertises
`mtp_num_hidden_layers=1`, but the released HF safetensors and upstream GGUF
files do not contain `mtp.*` / `nextn.*` tensors. To make MTP actually loadable,
this lab built a bootstrap MTP-only draft checkpoint:

- copied full-attention trunk layer 39 into `model.mtp.layers.0`
- added `model.mtp.fc.weight`, `model.mtp.pre_fc_norm_embedding.weight`, and
  `model.mtp.pre_fc_norm_hidden.weight`
- initialized `mtp.fc` to pass through the hidden-state half of
  `concat([embedding, hidden])`
- converted with `convert_hf_to_gguf.py --mtp`
- quantized the draft to IQ4_XS

This is a valid MTP serving artifact, but it is not a trained MTP head. Treat it
as an experimental bootstrap and training starting point, not as a production
speedup.

## Artifacts

Target:

- `artifacts/quant/ornith-1.0-35b-IQ4_XS.gguf`
- size: 18,939,311,680 bytes / 17.64 GiB

Bootstrap MTP draft:

- BF16 local draft: `artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-bf16.gguf`
- BF16 size: 3,735,537,248 bytes / 3.48 GiB
- IQ4_XS draft: `artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-IQ4_XS.gguf`
- IQ4_XS draft size: 1,149,036,128 bytes / 1.07 GiB

Builder:

```bash
.venv-train/bin/python scripts/build_bootstrap_mtp_checkpoint.py \
  --output artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39 \
  --source-layer 39 \
  --eh-proj-init hidden \
  --dtype bfloat16 \
  --force
```

Converter:

```bash
.venv-quant/bin/python vendor/llama.cpp/convert_hf_to_gguf.py \
  artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39 \
  --mtp \
  --outtype bf16 \
  --outfile artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-bf16.gguf
```

Draft quantizer:

```bash
vendor/llama.cpp/build-cpu/bin/llama-quantize \
  artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-bf16.gguf \
  artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-IQ4_XS.gguf \
  IQ4_XS 32
```

Quantizer output: source 3,552.05 MiB / 16.00 BPW; quantized 1,085.37 MiB /
4.89 BPW.

## Serving

Best-footprint MTP test profile:

```bash
QUANT=IQ4_XS \
SPEC_TYPE=draft-mtp \
SPEC_DRAFT_MODEL=artifacts/mtp/ornith-1.0-35b-mtp-bootstrap-layer39-IQ4_XS.gguf \
SPEC_DRAFT_N_MAX=1 \
SPEC_DRAFT_N_MIN=1 \
CTX_SIZE=8192 PARALLEL=16 BATCH_SIZE=4096 UBATCH_SIZE=512 \
REASONING=off \
scripts/serve_llamacpp_gpu0.sh
```

Loaded VRAM after benchmark warmup:

| Profile | Loaded VRAM MiB | Loaded VRAM GiB |
|---|---:|---:|
| IQ4_XS target only | 19,806 | 19.34 |
| IQ4_XS target + BF16 MTP draft, n_max=3 | 25,724 | 25.12 |
| IQ4_XS target + BF16 MTP draft, n_max=1 | 23,580 | 23.03 |
| IQ4_XS target + IQ4_XS MTP draft, n_max=1 | 21,832 | 21.32 |

## Correctness

All MTP profiles below passed the OpenAI-compatible correctness gate:

- `/v1/models` listed the served alias
- `/v1/chat/completions` returned coherent code
- `/v1/completions` returned coherent text
- output matched `def add(` and `return a + b`

Raw correctness probes:

- `runs/probe-llamacpp-iq4-xs-mtp-bootstrap-correctness.json`
- `runs/probe-llamacpp-iq4-xs-mtp-bootstrap-nmax1-correctness.json`
- `runs/probe-llamacpp-iq4-xs-mtp-bootstrap-iq4draft-nmax1-correctness.json`

## Throughput

Values are aggregate output tokens/second from
`scripts/bench_openai.py --stream --max-tokens 256`; each row had 0 failed
requests.

| Profile | c1 tok/s | c1 p95 TTFT ms | c4 tok/s | c4 p95 TTFT ms | c8 tok/s | c8 p95 TTFT ms | c16 tok/s | c16 p95 TTFT ms |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| IQ4_XS target only | 234.1 | 75.1 | 297.7 | 159.7 | 411.5 | 330.2 | 476.0 | 541.5 |
| IQ4_XS + BF16 MTP draft, n_max=3 | 95.5 | 84.3 | 97.0 | 179.2 | 102.1 | 408.1 | 108.2 | 726.8 |
| IQ4_XS + BF16 MTP draft, n_max=1 | 153.6 | 81.9 | 147.2 | 168.4 | 154.3 | 390.7 | 159.5 | 701.3 |
| IQ4_XS + IQ4_XS MTP draft, n_max=1 | 155.1 | 82.8 | 151.6 | 213.9 | 155.8 | 377.8 | 162.7 | 670.9 |

## Draft Acceptance

| Profile | Accepted draft tokens | Generated draft tokens | Acceptance |
|---|---:|---:|---:|
| BF16 MTP draft, n_max=3 | 195 | 41,577 | 0.47% |
| BF16 MTP draft, n_max=1 | 151 | 14,060 | 1.07% |
| IQ4_XS MTP draft, n_max=1 | 172 | 14,215 | 1.21% |

The bootstrap MTP head is therefore slower than target-only decoding. The
runtime and artifact path are now working, but the draft head must be trained
before MTP can become a serving speedup.

## Raw Artifacts

- `runs/build-mtp-bootstrap-layer39.log`
- `runs/convert-mtp-bootstrap-layer39.log`
- `runs/quant-mtp-bootstrap-layer39-iq4-xs.log`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-c1-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-c4-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-c8-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-c16-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-nmax1-c1-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-nmax1-c4-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-nmax1-c8-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-nmax1-c16-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-iq4draft-nmax1-c1-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-iq4draft-nmax1-c4-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-iq4draft-nmax1-c8-256.jsonl`
- `runs/llamacpp-iq4-xs-mtp-bootstrap-iq4draft-nmax1-c16-256.jsonl`