Upload L4_uniform (compressed) from MiniLM-L12
Browse files
README.md
CHANGED
|
@@ -103,6 +103,56 @@ embeddings = model.encode(sentences)
|
|
| 103 |
print(embeddings.shape) # (3, 384)
|
| 104 |
```
|
| 105 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
|
| 107 |
|
| 108 |
## Training
|
|
|
|
| 103 |
print(embeddings.shape) # (3, 384)
|
| 104 |
```
|
| 105 |
|
| 106 |
+
## MTEB Evaluation Results
|
| 107 |
+
|
| 108 |
+
**Overall Average: 49.02%**
|
| 109 |
+
|
| 110 |
+
| Task Group | Average |
|
| 111 |
+
|---|---|
|
| 112 |
+
| Classification | 56.87% |
|
| 113 |
+
| Clustering | 32.04% |
|
| 114 |
+
| STS | 57.15% |
|
| 115 |
+
|
| 116 |
+
### Classification
|
| 117 |
+
|
| 118 |
+
| Task | Average | Details |
|
| 119 |
+
|---|---|---|
|
| 120 |
+
| AmazonCounterfactualClassification | 67.02% | en: 70.31%, en-ext: 68.1%, de: 65.73% |
|
| 121 |
+
| Banking77Classification | 69.18% | default: 69.18% |
|
| 122 |
+
| ImdbClassification | 59.38% | default: 59.38% |
|
| 123 |
+
| MTOPDomainClassification | 71.48% | en: 80.02%, es: 73.78%, hi: 71.07% |
|
| 124 |
+
| MassiveIntentClassification | 36.9% | en: 58.41%, zh-CN: 58.07%, ja: 56.73% |
|
| 125 |
+
| MassiveScenarioClassification | 39.51% | zh-CN: 63.96%, en: 62.71%, ja: 59.84% |
|
| 126 |
+
| ToxicConversationsClassification | 62.02% | default: 62.02% |
|
| 127 |
+
| TweetSentimentExtractionClassification | 49.43% | default: 49.43% |
|
| 128 |
+
|
| 129 |
+
### Clustering
|
| 130 |
+
|
| 131 |
+
| Task | Average | Details |
|
| 132 |
+
|---|---|---|
|
| 133 |
+
| ArXivHierarchicalClusteringP2P | 49.93% | default: 49.93% |
|
| 134 |
+
| ArXivHierarchicalClusteringS2S | 46.08% | default: 46.08% |
|
| 135 |
+
| BiorxivClusteringP2P.v2 | 21.47% | default: 21.47% |
|
| 136 |
+
| MedrxivClusteringP2P.v2 | 26.05% | default: 26.05% |
|
| 137 |
+
| MedrxivClusteringS2S.v2 | 22.94% | default: 22.94% |
|
| 138 |
+
| StackExchangeClustering.v2 | 41.23% | default: 41.23% |
|
| 139 |
+
| StackExchangeClusteringP2P.v2 | 32.19% | default: 32.19% |
|
| 140 |
+
| TwentyNewsgroupsClustering.v2 | 16.43% | default: 16.43% |
|
| 141 |
+
|
| 142 |
+
### STS
|
| 143 |
+
|
| 144 |
+
| Task | Average | Details |
|
| 145 |
+
|---|---|---|
|
| 146 |
+
| BIOSSES | 45.64% | default: 45.64% |
|
| 147 |
+
| SICK-R | 62.01% | default: 62.01% |
|
| 148 |
+
| STS12 | 57.85% | default: 57.85% |
|
| 149 |
+
| STS13 | 65.48% | default: 65.48% |
|
| 150 |
+
| STS14 | 60.39% | default: 60.39% |
|
| 151 |
+
| STS15 | 73.93% | default: 73.93% |
|
| 152 |
+
| STS17 | 46.29% | en-en: 76.54%, es-es: 75.88%, ko-ko: 62.72% |
|
| 153 |
+
| STS22.v2 | 37.34% | zh: 57.86%, es: 54.85%, fr: 51.41% |
|
| 154 |
+
| STSBenchmark | 65.38% | default: 65.38% |
|
| 155 |
+
|
| 156 |
|
| 157 |
|
| 158 |
## Training
|