---
license: mit
tags:
  - onnx
  - embeddings
  - sentence-transformers
  - Octen
---

# Octen-Embedding-0.6B ONNX (INT4)

ONNX export of [Octen/Octen-Embedding-0.6B](https://huggingface.co/Octen/Octen-Embedding-0.6B) for inference with ONNX Runtime. This repo contains **only the INT4 variant** for smaller downloads. Other variants: [fp16](https://huggingface.co/geoffsee/octen-embedding-0.6b-onnx-fp16), [int8](https://huggingface.co/geoffsee/octen-embedding-0.6b-onnx-int8), [int4](https://huggingface.co/geoffsee/octen-embedding-0.6b-onnx-int4).

- **Base model:** Octen/Octen-Embedding-0.6B
- **Max sequence length:** 512
- **Hidden size:** 1024

## Files

| File | Description |
|------|-------------|
| `model.int4.onnx` (+ `.onnx.data`) | INT4 Bnb4 weight-only, ~537 MB |
| `tokenizer/` | Same as base model |

## Usage

```python
from huggingface_hub import hf_hub_download
import onnxruntime as ort
from transformers import AutoTokenizer

repo_id = "geoffsee/octen-embedding-0.6b-onnx-int4"
path = hf_hub_download(repo_id=repo_id, filename="model.int4.onnx")
tokenizer = AutoTokenizer.from_pretrained(repo_id)
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])
# ... encode and run ...
```