tardellirs commited on
Commit
8840bc8
Β·
1 Parent(s): 1416cf3

Rebrand headings to MTEB-BR (h1 'MTEB-BR Brazilian Portuguese Leaderboard'; title/desc/card MTEB(por) -> MTEB-BR)

Browse files
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: MTEB(por) Leaderboard
3
  emoji: πŸ†
4
  colorFrom: green
5
  colorTo: yellow
@@ -8,102 +8,73 @@ pinned: true
8
  license: apache-2.0
9
  short_description: Massive Text Embedding Benchmark for Brazilian Portuguese
10
  tags:
11
- - leaderboard
12
- - benchmark
13
- - embeddings
14
- - portuguese
15
- - brazilian-portuguese
16
- - mteb
17
- - retrieval
18
- - sentence-transformers
19
  datasets:
20
- - MTEB-BR/mteb-pt-results
21
  models:
22
- - Alibaba-NLP/gte-Qwen2-1.5B-instruct
23
- - Alibaba-NLP/gte-Qwen2-7B-instruct
24
- - BAAI/bge-m3
25
- - BAAI/bge-small-en-v1.5
26
- - BidirLM/BidirLM-0.6B-Embedding
27
- - BidirLM/BidirLM-1.7B-Embedding
28
- - BidirLM/BidirLM-1B-Embedding
29
- - ICT-TIME-and-Querit/BOOM_4B_v1
30
- - Linq-AI-Research/Linq-Embed-Mistral
31
- - MTEB-BR/baseline-random-encoder
32
- - Mihaiii/Ivysaur
33
- - Octen/Octen-Embedding-0.6B
34
- - Octen/Octen-Embedding-8B
35
- - PORTULAN/albertina-900m-portuguese-ptbr-encoder
36
- - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder
37
- - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder-ir
38
- - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder
39
- - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir
40
- - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder
41
- - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder-ir
42
- - Qwen/Qwen3-Embedding-0.6B
43
- - Qwen/Qwen3-Embedding-4B
44
- - Qwen/Qwen3-Embedding-8B
45
- - Salesforce/SFR-Embedding-2_R
46
- - Salesforce/SFR-Embedding-Mistral
47
- - SamilPwC-AXNode-GenAI/PwC-Embedding_expr
48
- - Snowflake/snowflake-arctic-embed-l-v2.0
49
- - avsolatorio/GIST-all-MiniLM-L6-v2
50
- - avsolatorio/NoInstruct-small-Embedding-v0
51
- - codefuse-ai/F2LLM-0.6B
52
- - codefuse-ai/F2LLM-v2-0.6B
53
- - codefuse-ai/F2LLM-v2-1.7B
54
- - codefuse-ai/F2LLM-v2-14B
55
- - codefuse-ai/F2LLM-v2-160M
56
- - codefuse-ai/F2LLM-v2-330M
57
- - codefuse-ai/F2LLM-v2-4B
58
- - codefuse-ai/F2LLM-v2-80M
59
- - codefuse-ai/F2LLM-v2-8B
60
- - google/embeddinggemma-300m
61
- - ibm-granite/granite-embedding-107m-multilingual
62
- - ibm-granite/granite-embedding-311m-multilingual-r2
63
- - ibm-granite/granite-embedding-97m-multilingual-r2
64
- - intfloat/e5-mistral-7b-instruct
65
- - intfloat/e5-small-v2
66
- - intfloat/multilingual-e5-base
67
- - intfloat/multilingual-e5-large
68
- - intfloat/multilingual-e5-large-instruct
69
- - intfloat/multilingual-e5-small
70
- - jinaai/jina-embeddings-v5-text-small
71
- - lfcc/medlink-bi-encoder
72
- - microsoft/harrier-oss-v1-0.6b
73
- - microsoft/harrier-oss-v1-270m
74
- - microsoft/harrier-oss-v1-27b
75
- - mixedbread-ai/mxbai-embed-large-v1
76
- - neuralmind/bert-base-portuguese-cased
77
- - neuralmind/bert-large-portuguese-cased
78
- - nvidia/llama-embed-nemotron-8b
79
- - rufimelo/Legal-BERTimbau-sts-large
80
- - rufimelo/Legal-BERTimbau-sts-large-ma-v3
81
- - sentence-transformers/LaBSE
82
- - sentence-transformers/all-MiniLM-L12-v2
83
- - sentence-transformers/all-MiniLM-L6-v2
84
- - sentence-transformers/all-mpnet-base-v2
85
- - sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
86
- - sentence-transformers/paraphrase-multilingual-mpnet-base-v2
87
- - stjiris/bert-large-portuguese-cased-legal-mlm-mkd-nli-sts-v1
88
- - stjiris/bert-large-portuguese-cased-legal-mlm-sts-v1.0
89
- - stjiris/bert-large-portuguese-cased-legal-tsdae-gpl-nli-sts-MetaKD-v0
90
- - tardellirs/brazembed-pt-br
91
- - telepix/PIXIE-Rune-v1.0
92
- - tencent/KaLM-Embedding-Gemma3-12B-2511
93
- - thenlper/gte-small
94
- - ufca-llms/Qwen3-Embedding-0.6B-jua-V2
95
- - ufca-llms/jua-4B-legal-only
96
- - ufca-llms/jua-4B-mixed
97
- - ulysses-camara/legal-bert-pt-br
98
  ---
99
 
100
- # MTEB(por) β€” Brazilian Portuguese Massive Text Embedding Benchmark
101
 
102
  A Massive Text Embedding Benchmark for **Brazilian Portuguese**.
103
 
104
- - **93 models** evaluated on **22 native PT-BR tasks** (no machine-translated mMARCO/mkqa)
105
  - Tasks: classification, NLI, STS, clustering, retrieval (incl. Quati Pool), reranking
106
- - Headline metric: **mean_22** (average across all 22 tasks)
107
  - All raw results: [MTEB-BR/mteb-pt-results](https://huggingface.co/datasets/MTEB-BR/mteb-pt-results)
108
  - Source code: [github.com/tardellirs/mteb-br](https://github.com/tardellirs/mteb-br)
109
 
@@ -127,7 +98,7 @@ with model_id, eval JSONs, and reproduction script.
127
 
128
  ```bibtex
129
  @misc{mteb-por-2026,
130
- title = {MTEB(por): The Brazilian Portuguese MTEB Sub-Benchmark},
131
  author = {Tardelli, R.S.},
132
  year = {2026},
133
  url = {https://huggingface.co/spaces/MTEB-BR/leaderboard}
 
1
  ---
2
+ title: MTEB-BR Leaderboard
3
  emoji: πŸ†
4
  colorFrom: green
5
  colorTo: yellow
 
8
  license: apache-2.0
9
  short_description: Massive Text Embedding Benchmark for Brazilian Portuguese
10
  tags:
11
+ - leaderboard
12
+ - benchmark
13
+ - embeddings
14
+ - portuguese
15
+ - brazilian-portuguese
16
+ - mteb
17
+ - retrieval
18
+ - sentence-transformers
19
  datasets:
20
+ - MTEB-BR/mteb-pt-results
21
  models:
22
+ - Qwen/Qwen3-Embedding-8B
23
+ - Octen/Octen-Embedding-8B
24
+ - google/embeddinggemma-300m
25
+ - microsoft/harrier-oss-v1-27b
26
+ - Qwen/Qwen3-Embedding-4B
27
+ - tencent/KaLM-Embedding-Gemma3-12B-2511
28
+ - codefuse-ai/F2LLM-v2-8B
29
+ - microsoft/harrier-oss-v1-0.6b
30
+ - codefuse-ai/F2LLM-v2-14B
31
+ - intfloat/multilingual-e5-large-instruct
32
+ - SamilPwC-AXNode-GenAI/PwC-Embedding_expr
33
+ - BAAI/bge-m3
34
+ - Qwen/Qwen3-Embedding-0.6B
35
+ - telepix/PIXIE-Rune-v1.0
36
+ - ufca-llms/jua-4B-mixed
37
+ - Octen/Octen-Embedding-0.6B
38
+ - jinaai/jina-embeddings-v5-text-small
39
+ - intfloat/multilingual-e5-large
40
+ - Snowflake/snowflake-arctic-embed-l-v2.0
41
+ - ibm-granite/granite-embedding-311m-multilingual-r2
42
+ - intfloat/multilingual-e5-base
43
+ - PORTULAN/serafim-335m-portuguese-pt-sentence-encoder-ir
44
+ - PORTULAN/serafim-100m-portuguese-pt-sentence-encoder-ir
45
+ - PORTULAN/serafim-900m-portuguese-pt-sentence-encoder-ir
46
+ - microsoft/harrier-oss-v1-270m
47
+ - jinaai/jina-embeddings-v5-text-nano
48
+ - intfloat/multilingual-e5-small
49
+ - ibm-granite/granite-embedding-107m-multilingual
50
+ - codefuse-ai/F2LLM-v2-160M
51
+ - rufimelo/Legal-BERTimbau-sts-large
52
+ - ibm-granite/granite-embedding-97m-multilingual-r2
53
+ - Alibaba-NLP/gte-Qwen2-1.5B-instruct
54
+ - sentence-transformers/paraphrase-multilingual-mpnet-base-v2
55
+ - stjiris/bert-large-portuguese-cased-legal-mlm-sts-v1.0
56
+ - sentence-transformers/LaBSE
57
+ - rufimelo/Legal-BERTimbau-sts-large-ma-v3
58
+ - neuralmind/bert-base-portuguese-cased
59
+ - neuralmind/bert-large-portuguese-cased
60
+ - mixedbread-ai/mxbai-embed-large-v1
61
+ - PORTULAN/albertina-900m-portuguese-ptbr-encoder
62
+ - sentence-transformers/all-MiniLM-L12-v2
63
+ - nvidia/llama-embed-nemotron-8b
64
+ - PORTULAN/albertina-100m-portuguese-ptbr-encoder
65
+ - sentence-transformers/all-mpnet-base-v2
66
+ - sentence-transformers/all-MiniLM-L6-v2
67
+ - BidirLM/BidirLM-1.7B-Embedding
68
+ - sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  ---
70
 
71
+ # MTEB-BR β€” Brazilian Portuguese Massive Text Embedding Benchmark
72
 
73
  A Massive Text Embedding Benchmark for **Brazilian Portuguese**.
74
 
75
+ - **56 models** evaluated on **16 native PT-BR tasks** (no machine-translated mMARCO/mkqa)
76
  - Tasks: classification, NLI, STS, clustering, retrieval (incl. Quati Pool), reranking
77
+ - Headline metric: **mean_16** (average across all 16 tasks)
78
  - All raw results: [MTEB-BR/mteb-pt-results](https://huggingface.co/datasets/MTEB-BR/mteb-pt-results)
79
  - Source code: [github.com/tardellirs/mteb-br](https://github.com/tardellirs/mteb-br)
80
 
 
98
 
99
  ```bibtex
100
  @misc{mteb-por-2026,
101
+ title = {MTEB-BR: The Brazilian Portuguese MTEB Sub-Benchmark},
102
  author = {Tardelli, R.S.},
103
  year = {2026},
104
  url = {https://huggingface.co/spaces/MTEB-BR/leaderboard}
assets/{index-B7JlBoRm.js β†’ index-VP1v9aXa.js} RENAMED
The diff for this file is too large to render. See raw diff
 
index.html CHANGED
@@ -4,11 +4,11 @@
4
  <meta charset="UTF-8" />
5
  <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
- <title>MTEB(por) Leaderboard β€” Brazilian Portuguese</title>
8
- <meta name="description" content="MTEB(por) β€” Massive Text Embedding Benchmark for Brazilian Portuguese. 93 models Γ— 22 tasks." />
9
- <meta property="og:title" content="MTEB(por) Leaderboard" />
10
  <meta property="og:description" content="93 embedding models evaluated on 22 native PT-BR tasks." />
11
- <script type="module" crossorigin src="/assets/index-B7JlBoRm.js"></script>
12
  <link rel="stylesheet" crossorigin href="/assets/index-igOVgDnG.css">
13
  </head>
14
  <body>
 
4
  <meta charset="UTF-8" />
5
  <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
+ <title>MTEB-BR Leaderboard β€” Brazilian Portuguese</title>
8
+ <meta name="description" content="MTEB-BR β€” Massive Text Embedding Benchmark for Brazilian Portuguese. 93 models Γ— 22 tasks." />
9
+ <meta property="og:title" content="MTEB-BR Leaderboard" />
10
  <meta property="og:description" content="93 embedding models evaluated on 22 native PT-BR tasks." />
11
+ <script type="module" crossorigin src="/assets/index-VP1v9aXa.js"></script>
12
  <link rel="stylesheet" crossorigin href="/assets/index-igOVgDnG.css">
13
  </head>
14
  <body>