README.md

metadata

title: MTEB-BR Leaderboard
emoji: 🏆
colorFrom: green
colorTo: yellow
sdk: static
pinned: true
license: apache-2.0
short_description: Massive Text Embedding Benchmark for Brazilian Portuguese
thumbnail: https://mteb-br.org/img/og.png
tags:
  - leaderboard
  - benchmark
  - embeddings
  - portuguese
  - brazilian-portuguese
  - mteb
  - retrieval
  - sentence-transformers
datasets:
  - MTEB-BR/mteb-pt-results
models:
  - Qwen/Qwen3-Embedding-8B
  - tencent/KaLM-Embedding-Gemma3-12B-2511
  - Octen/Octen-Embedding-8B
  - Qwen/Qwen3-Embedding-4B
  - Salesforce/SFR-Embedding-Mistral
  - BidirLM/BidirLM-1.7B-Embedding
  - ICT-TIME-and-Querit/BOOM_4B_v1
  - google/embeddinggemma-300m
  - Linq-AI-Research/Linq-Embed-Mistral
  - jinaai/jina-embeddings-v5-text-small
  - intfloat/multilingual-e5-large-instruct
  - Salesforce/SFR-Embedding-2_R
  - Alibaba-NLP/gte-Qwen2-7B-instruct
  - microsoft/harrier-oss-v1-27b
  - BidirLM/BidirLM-1B-Embedding
  - codefuse-ai/F2LLM-v2-8B
  - microsoft/harrier-oss-v1-0.6b
  - codefuse-ai/F2LLM-v2-14B
  - codefuse-ai/F2LLM-v2-4B
  - SamilPwC-AXNode-GenAI/PwC-Embedding_expr
  - Octen/Octen-Embedding-0.6B
  - telepix/PIXIE-Rune-v1.0
  - Qwen/Qwen3-Embedding-0.6B
  - Alibaba-NLP/gte-Qwen2-1.5B-instruct
  - Snowflake/snowflake-arctic-embed-l-v2.0
  - BAAI/bge-m3
  - codefuse-ai/F2LLM-v2-1.7B
  - BidirLM/BidirLM-0.6B-Embedding
  - ufca-llms/jua-4B-mixed
  - codefuse-ai/F2LLM-0.6B
  - microsoft/harrier-oss-v1-270m
  - codefuse-ai/F2LLM-v2-0.6B
  - intfloat/multilingual-e5-large
  - ibm-granite/granite-embedding-311m-multilingual-r2
  - ufca-llms/jua-4B-legal-only
  - codefuse-ai/F2LLM-v2-330M
  - intfloat/multilingual-e5-base
  - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder-ir
  - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir
  - intfloat/multilingual-e5-small
  - ibm-granite/granite-embedding-107m-multilingual
  - ibm-granite/granite-embedding-97m-multilingual-r2
  - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder-ir
  - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder
  - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder
  - codefuse-ai/F2LLM-v2-160M
  - sentence-transformers/paraphrase-multilingual-mpnet-base-v2
  - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder
  - ufca-llms/Qwen3-Embedding-0.6B-jua-V2
  - codefuse-ai/F2LLM-v2-80M
  - sentence-transformers/LaBSE
  - lfcc/medlink-bi-encoder
  - intfloat/e5-small-v2
  - mixedbread-ai/mxbai-embed-large-v1
  - BAAI/bge-small-en-v1.5
  - neuralmind/bert-large-portuguese-cased
  - thenlper/gte-small
  - neuralmind/bert-base-portuguese-cased
  - stjiris/bert-large-portuguese-cased-legal-mlm-sts-v1.0
  - intfloat/e5-mistral-7b-instruct
  - nvidia/llama-embed-nemotron-8b
  - rufimelo/Legal-BERTimbau-sts-large
  - stjiris/bert-large-portuguese-cased-legal-mlm-mkd-nli-sts-v1
  - Mihaiii/Ivysaur
  - PORTULAN/albertina-900m-portuguese-ptbr-encoder
  - sentence-transformers/all-MiniLM-L12-v2
  - avsolatorio/GIST-all-MiniLM-L6-v2
  - sentence-transformers/all-mpnet-base-v2
  - ulysses-camara/legal-bert-pt-br
  - sentence-transformers/all-MiniLM-L6-v2
  - rufimelo/Legal-BERTimbau-sts-large-ma-v3
  - stjiris/bert-large-portuguese-cased-legal-tsdae-gpl-nli-sts-MetaKD-v0
  - sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

MTEB-BR — Brazilian Portuguese Massive Text Embedding Benchmark

A Massive Text Embedding Benchmark for Brazilian Portuguese — native, not translated.

• 93 models (73 open-weight + 20 closed commercial APIs) evaluated on 22 native Brazilian-Portuguese tasks across 7 categories
• Admits only data created or found in Portuguese; machine-translated benchmarks (e.g. mMARCO, mkqa) are excluded by construction
• Headline metric: mean_22 (average across all 22 tasks), reported with per-task bootstrap confidence intervals, paired-bootstrap significance, and IRT task discrimination
• Website: mteb-br.org · Paper: arXiv:2607.04581
• All raw results: MTEB-BR/mteb-pt-results · Source code: github.com/tardellirs/mteb-br

Tasks (22)

Category	Tasks
Classification	HateBR, ToxSyn-PT, FactckBR, PortuLexRRIP
Multi-label classification	BrighterEmotion
Pair classification	ASSIN-RTE, InferBR
Semantic textual similarity (STS)	ASSIN-STS, ASSIN2-STS
Clustering	WikipediaPT-Categories, MedPT, JurisTCU, SciELO, StackOverflow-PT
Retrieval	Quati, JurisTCU, BRTaxQA, FaQuAD, MedPT, FaQ-Bacen
Reranking	Quati, JurisTCU

How to add a model

Submit via GitHub Issues at tardellirs/mteb-br/issues with the model ID, evaluation JSONs, and a reproduction script.

Citation

@article{stekel2026mtebbr,
  title   = {MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese},
  author  = {Stekel, Tardelli Ronan Coelho},
  journal = {arXiv preprint arXiv:2607.04581},
  year    = {2026}
}