# opencoti llamafile 0.10.3-c3

Third packaged cut (`c` series) of the opencoti single-file inference
engine (#613 lane). Built from
[Mozilla-Ocho/llamafile v0.10.3](https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.10.3)
+ the opencoti patch series (79 patches, `0001`–`0133`) at
`vendors/patches/llamafile/`.

Version string: `opencoti-0.10.3-c3` — printed by `--version` and exposed
at runtime.

## Artifacts

| File | Size | sha256 |
|---|---:|---|
| `opencoti-llamafile-0.10.3-c3-x86_64.llamafile` | 4,812,296,839 B | `8a35b1f87b7bb7f2915d8756450f09008f4cd8331cf4a763e40edbcb8c4cf7e9` |
| `opencoti-llamafile-0.10.3-c3-win-x86_64.llamafile.exe` | 55,429,522 B | `e5aa9a3d4e8274ca430f3a5557708955821cbc2d6d2c20f36a1c24228b0c642d` |
| `dso/0.10.3-c3/ggml-cuda-x86_64.so` (side-load DSO, HF only) | 4,756,838,104 B | `5e0342ba9b2ded4847280067dfd6ebc4257e1b83a984881988c528c163b23e4a` |

The main artifact's embedded `ggml-cuda.so` is byte-identical to the
standalone side-load DSO above (glibc floor GLIBC_2.29 — runs on
Debian 11 / glibc 2.31 and newer).

Single-file Cosmopolitan APE. **Fat binary: runs natively on x86_64 and
aarch64** (Linux/macOS/Windows/BSD) — on ARM64 hosts (DGX Spark, GH200,
Apple Silicon) inference is CPU-only in this cut; the CUDA DSO is
x86_64-linux. GPU-enabled aarch64 (sbsa) is a planned separate
artifact.

### `-win` variant

The full artifact cannot run on Windows: **Windows refuses to execute
binaries larger than 4 GB**. `…-win-x86_64.llamafile.exe` is the same
host APE (identical patches/features, `.exe` extension Windows
requires) with **no embedded GPU backend** — 55 MB, CPU inference out
of the box on Windows/Linux/macOS/BSD. GPU:

- **Windows:** install CUDA Toolkit + MSVC; llamafile compiles
  `ggml-cuda.dll` on the fly on first `-ngl` run (upstream mechanism).
- **Linux x86_64:** side-load the standalone DSO:
  `mkdir -p ~/.llamafile/v/0.10.3/ && cp ggml-cuda-x86_64.so
  ~/.llamafile/v/0.10.3/ggml-cuda.so` (verify sha against
  `SHA256SUMS` first).

Run as a server:

```bash
sh ./opencoti-llamafile-0.10.3-c3-x86_64.llamafile --server --port 8080 \
    -m <model>.gguf -ngl 99 --flash-attn on
```

On first GPU run the embedded `ggml-cuda.so` (CUDA 13.2, arches
sm_75/80/86/89/90/120) self-extracts to `~/.llamafile/v/0.10.3/`.
Without `--server` the binary starts the interactive chat CLI.

## What's new vs c2 (2026-07-05)

**Packaging:**
- NEW `-win` variant (Windows-runnable bare APE, GPU via side-load)
  + standalone side-load CUDA DSO published under `dso/` on HF
- Supported/target model families (Gemma-4 primary, Qwen secondary)
  now documented prominently in README + USAGE

**Host binary (patches 0122–0133):**
- RYS `--repeat-layers` weight-shared layer duplication + `--rys-probe`
  v2 task battery (#656–#673)
- bug-858 CLOSED: MTP speculative decode at/above upstream-b9859 parity
  on both native pairs (spec-clone-cheap + verify GEMM lift, 0128)
- P2 mixed-KV auto-tail + parallel≥2 fit/checkpoint fixes (0121, 0130)
- Runtime introspection (#676/#677, 0132): `/props` gains an `"opencoti"`
  section (kv, residency, dca, speculative, sparse_attn, kv_reuse,
  repeat_layers, rest_kv); `/slots` gains per-slot lifetime draft
  counters + session/pool binding
- Full `--version` string `opencoti-<ver>-<tag>` (#613, 0133)

**CUDA DSO (rebuilt, 4.4 GB):**
- #674: DCA-off scalar-MMA tensor-core decode generalized to D256/D512
  (Gemma-4 head dims) — q8q4 KV decode ≈1.15× f16, up to 2.3× native VEC
- 0128 mmvf f16/bf16 TinyBLAS verify lift (Qwen NextN n3 parity)
- 0126/0127 P0 DCA-off scalar-MMA D128 foundation

## Verification

`MANIFEST.json` ships alongside the artifact: versionString, gitCommit,
full patch list, and per-DSO sha256 of the embedded backend. Verify the
embedded DSO without running:

```bash
unzip -p opencoti-llamafile-0.10.3-c3-x86_64.llamafile ggml-cuda.so | sha256sum
```

Zero-dependency smoke performed on this cut: isolated-`HOME` `--version`,
DSO self-extraction sha-match, `--server` GPU boot on RTX 3090 with
`/props` opencoti introspection + greedy completion.

## Build provenance

- Repo: `mann1x/opencoti` @ commit `4c3f2ec2e` (dev branch)
- Base: llamafile 0.10.3 (outer 7fca8b2, nested llama.cpp dbe9c0c) +
  Mozilla patch overlay + opencoti chain 0001–0133 (replay-proven
  byte-identical to the shipped tree)
- Host binary built on solidPC (cosmocc); CUDA DSO built on solidPC
  (Debian 11, CUDA 13.2) to pin the release glibc floor at 2.29
  (bug-2190: a bs2-built DSO required glibc ≥2.32 and failed on
  older hosts)

## License

Apache-2.0 (Mozilla-Ocho/llamafile) + MIT (llama.cpp and bundled
projects). opencoti patches and packaging are MIT.