---
base_model: lightonai/LateOn
library_name: litembeddings
license: apache-2.0
language:
- en
pipeline_tag: sentence-similarity
tags:
- gguf
- litembeddings
- modernbert
- colbert
- late-interaction
- pylate
- retrieval
- quantized
---

# LateOn Q8_0 for litembeddings

A Q8_0 GGUF conversion of [lightonai/LateOn](https://huggingface.co/lightonai/LateOn), a 149M-parameter ModernBERT ColBERT retrieval model. This package is built for [litembeddings](https://github.com/alexandernicholson/litembeddings) and includes the model's complete three-stage PyLate projection folded into one 768-to-128 matrix.

## Files

| File | Purpose | SHA-256 |
|---|---|---|
| `LateOn-Q8_0.gguf` | Q8_0 ModernBERT encoder | `fe917ea8a55c97a847bbdac1d950053e6fa0fab7a571ae77cd39480198a15290` |
| `LateOn-Q8_0.projection` | Folded 768-to-128 PyLate projection | `3e7fd24683cbcde958448d2d7120d7f12d4d0abcb67bed47e19e7a3170a5fa0a` |

Both files are required. The GGUF encoder alone produces 768-dimensional token vectors; litembeddings applies and normalizes the supplied projection to produce the model's intended 128-dimensional token vectors.

## litembeddings usage

```sql
.load ./litembeddings

SELECT lembed_model(
  '/path/to/LateOn-Q8_0.gguf',
  json_object(
    'colbert_projection', '/path/to/LateOn-Q8_0.projection',
    'ctx_size', 300,
    'batch_size', 300
  )
);

-- Full-precision token vectors.
SELECT lembed_tokens('What is SQLite WAL mode?');

-- Compact int8 token vectors for storage and MaxSim search.
SELECT lembed_tokens_quantized('What is SQLite WAL mode?');

SELECT lembed_maxsim(
  lembed_tokens('What is SQLite WAL mode?'),
  lembed_tokens('WAL mode lets SQLite readers coexist with one writer.')
);
```

LateOn was trained for queries up to 32 tokens and documents up to 300 tokens. Use the same preprocessing and retrieval conventions as the source model when comparing quality results.

## Conversion provenance

- Source: [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn)
- Source revision: `c01907b70557ee5c7753680d4819a5cce1674b83`
- Converted: 2026-07-14
- llama.cpp revision: `6eddde06a4f25d55d538b5d15628dcc2b6882147`
- Quantization: Q8_0
- Projection: `W3 @ (W2 + R2) @ (W1 + R1)`, preserving both learned dimension-changing PyLate residuals

An end-to-end parity check against the source FP32 Transformers/PyLate pipeline produced per-token cosine similarity `min=0.999883`, `mean=0.999951` for the validation text. Small differences are expected from Q8_0 quantization.

## License and attribution

The source model is released under Apache 2.0. See the [LateOn model card](https://huggingface.co/lightonai/LateOn) for training details, evaluation results, intended use, limitations, and citation information. This repository is an independent conversion and is not affiliated with LightOn.