Feature Extraction
sentence-transformers
ONNX
Transformers
modernbert
onnxruntime
reranker
int8
int4
text-embeddings-inference
# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("jrc2139/granite-embedding-small-english-r2-ONNX")
model = AutoModel.from_pretrained("jrc2139/granite-embedding-small-english-r2-ONNX", device_map="auto")Quick Links
ONNX Quantized versions of ibm-granite/granite-embedding-small-english-r2
This repository contains ONNX export and multiple quantized versions of ibm-granite/granite-embedding-small-english-r2.
Usage
from sentence_transformers import SentenceTransformer
# Load Int8 model (ARM64 example)
model = SentenceTransformer(
"jrc2139/granite-embedding-small-english-r2-ONNX",
backend="onnx",
model_kwargs={"file_name": "onnx/model_qint8_arm64.onnx"},
trust_remote_code=True
)
- Downloads last month
- 8
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="jrc2139/granite-embedding-small-english-r2-ONNX")