tooape/granite-embedding-97m-multilingual-r2-GBQ4-ONNX

granite-embedding-97m-multilingual-r2 — GBQ-int4 ONNX (transformers.js)

ONNX export of ibm-granite/granite-embedding-97m-multilingual-r2 for transformers.js / onnxruntime-web, sized for in-browser retrieval (61.2 MB):

• Body: MatMulNBits q4 (block 32, symmetric, accuracy_level 4)
• Embedding table (180k × 384, 71% of params): GatherBlockQuantized int4, quant_axes=("Gather", 1) — runs natively on the WebGPU EP (ORT ≥ 1.23); plain int8 Gather has no WebGPU kernel and falls back to WASM.
• Exported via optimum (feature-extraction) + MatMulNBitsQuantizer with op_types_to_quantize=("MatMul", "Gather"), opset 21.

Outputs token_embeddings / sentence_embedding; use pooling: 'cls', normalize: true (the model is CLS-pooled, symmetric — no query/doc prompts).

Quantization relevance gate (vs fp32, graded nDCG@10 on private retrieval corpora + Belebele): English ±0.003; Belebele dense −0.005 (en) … −0.021 (ko), multilingual advantage over English-only siblings fully intact.

import { pipeline } from '@huggingface/transformers';
const extractor = await pipeline('feature-extraction',
  'tooape/granite-embedding-97m-multilingual-r2-GBQ4-ONNX',
  { device: 'webgpu', dtype: 'q4' });
const v = await extractor('search query', { pooling: 'cls', normalize: true });