---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- multi-vector
- colbert
- late-interaction
- generated_from_trainer
- dataset_size:99000
- loss:CachedMultiVectorMultipleNegativesRankingLoss
base_model: answerdotai/ModernBERT-base
widget:
- text: 'In the United States, under current patent law, the term of patent, provided
that maintenance fees are paid on time, are: 1 For applications filed on or after
June 8, 1995, the patent term is 20 years from the filing date of the earliest
U.S. or international (PCT) application to which priority is claimed (excluding
provisional applications).'
- text: CBC with Differential Blood Test. CBC with Differential Blood Test. A Complete
Blood Count (CBC) with Differential is a broad screening test which can aid in
the diagnosis of a variety of conditions and diseases such as Anemia, Leukemia,
bleeding disorders, and infections.
- text: what are some vod platforms?
- text: "Types of antibiotics. There are hundreds of different types of antibiotics,\
\ but most of them can be broadly classified into six groups. These are outlined\
\ below. Penicillins (such as penicillin and amoxicillin) â\x80\x93 widely used\
\ to treat a variety of infections, including skin infections, chest infections\
\ and urinary tract infections."
- text: territorial sovereignty. Exclusive right of a state to exercise its powers
within the boundaries of its territory.
datasets:
- sentence-transformers/msmarco-bm25
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- maxsim_accuracy@1
- maxsim_accuracy@3
- maxsim_accuracy@5
- maxsim_accuracy@10
- maxsim_precision@1
- maxsim_precision@3
- maxsim_precision@5
- maxsim_precision@10
- maxsim_recall@1
- maxsim_recall@3
- maxsim_recall@5
- maxsim_recall@10
- maxsim_ndcg@10
- maxsim_mrr@10
- maxsim_map@100
model-index:
- name: ColBERT ModernBERT-base trained on MS MARCO triplets with GradCache
results:
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: maxsim_accuracy@1
value: 0.32
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.5
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.58
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.7
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.32
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.16666666666666669
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.11599999999999999
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.32
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.5
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.58
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.7
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.5035525583268671
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.4419126984126984
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.4558177677456216
name: Maxsim Map@100
- type: maxsim_accuracy@1
value: 0.34
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.5
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.6
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.72
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.34
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.16666666666666663
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.12000000000000002
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07200000000000001
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.34
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.5
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.6
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.72
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.5196566650374043
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.4571904761904762
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.4702762122466456
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: maxsim_accuracy@1
value: 0.32
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.5
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.58
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.66
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.32
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.1733333333333333
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.12
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.068
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.3
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.49
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.56
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.62
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.4702309838483119
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.43272222222222223
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.42919136794674306
name: Maxsim Map@100
- type: maxsim_accuracy@1
value: 0.38
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.5
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.6
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.7
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.38
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.1733333333333333
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.124
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07200000000000001
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.36
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.49
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.58
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.65
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.4984814791818691
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.4622222222222223
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.4559346570489271
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: maxsim_accuracy@1
value: 0.32
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.38
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.44
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.52
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.32
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.16666666666666663
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.11599999999999999
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.076
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.21335714285714286
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.28488095238095235
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.31288095238095237
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.3684047619047619
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.32790029976167445
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.3730555555555556
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.2941689356621179
name: Maxsim Map@100
- type: maxsim_accuracy@1
value: 0.3
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.4
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.44
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.52
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.3
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.16666666666666663
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.124
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07800000000000001
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.19335714285714287
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.28488095238095235
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.3257380952380952
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.37240476190476185
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.32253820883537515
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.36407936507936506
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.28501846515826573
name: Maxsim Map@100
- task:
type: multi-vector-nano-beir
name: Multi Vector Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: maxsim_accuracy@1
value: 0.32
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.45999999999999996
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.5333333333333333
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.6266666666666666
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.32
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.16888888888888887
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.11733333333333333
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07133333333333335
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.2777857142857143
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.4249603174603174
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.48429365079365083
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.5628015873015872
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.4338946139789512
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.41589682539682543
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.39305935711816087
name: Maxsim Map@100
- type: maxsim_accuracy@1
value: 0.4345368916797488
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.6119309262166405
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.6766718995290424
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.758430141287284
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.4345368916797488
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.2709576138147567
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.2108131868131868
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.14876609105180533
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.2568503828650686
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.39219839167176673
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.4528025716195824
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.526628596165496
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.47744189955965444
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.5363376442151953
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.40664976433960326
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: maxsim_accuracy@1
value: 0.2
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.38
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.46
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.54
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.2
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.12666666666666665
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.09600000000000002
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.064
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.09166666666666666
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.17066666666666666
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.20633333333333334
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.27033333333333337
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.2154158108818191
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.3006904761904762
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.165789222440451
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: maxsim_accuracy@1
value: 0.56
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.78
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.82
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.86
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.56
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.5
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.44800000000000006
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.3739999999999999
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.06463297360998158
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.1224040434115415
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.17209776544895988
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.24843171098171568
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.4665833494159522
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.6738571428571427
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.3212853411213698
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: maxsim_accuracy@1
value: 0.58
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.9
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.94
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.98
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.58
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.30666666666666664
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.19199999999999995
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.102
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.5466666666666667
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.8466666666666667
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.8866666666666667
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.9333333333333332
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.7634632906900782
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.7285238095238096
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.6972606027433613
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: maxsim_accuracy@1
value: 0.68
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.76
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.78
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.84
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.68
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.28
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.19199999999999995
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.10799999999999997
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.34
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.42
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.48
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.54
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.5293806892763252
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.7263888888888889
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.4467598142471342
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: maxsim_accuracy@1
value: 0.42
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.5
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.54
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.56
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.42
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.33333333333333337
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.29600000000000004
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.23399999999999999
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.027922117605102565
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.09673465072936029
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.11351894804424907
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.1360204373302154
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.30660127324494973
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.4661904761904762
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.14095019453849242
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: maxsim_accuracy@1
value: 0.78
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.88
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.9
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.94
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.78
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.3666666666666666
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.23199999999999996
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.12399999999999999
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.6806666666666666
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.8353333333333333
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.872
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.9226666666666666
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.8471248754431309
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.8372222222222221
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.8197089910089909
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: maxsim_accuracy@1
value: 0.34
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.54
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.62
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.76
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.34
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.26666666666666666
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.21599999999999997
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.152
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.07166666666666667
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.16466666666666666
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.22166666666666668
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.31066666666666665
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.2973905321869307
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.4649920634920634
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.22650133778044473
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: maxsim_accuracy@1
value: 0.14
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.46
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.62
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.76
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.14
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.15333333333333332
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.12400000000000003
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.07600000000000001
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.14
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.46
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.62
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.76
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.422866421857334
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.3170714285714286
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.3249369303386584
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: maxsim_accuracy@1
value: 0.48
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.6
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.64
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.7
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.48
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.22666666666666668
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.14800000000000002
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.08
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.445
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.595
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.64
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.7
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.5831859060929877
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.5505238095238095
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.5527097251437221
name: Maxsim Map@100
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: maxsim_accuracy@1
value: 0.4489795918367347
name: Maxsim Accuracy@1
- type: maxsim_accuracy@3
value: 0.7551020408163265
name: Maxsim Accuracy@3
- type: maxsim_accuracy@5
value: 0.8367346938775511
name: Maxsim Accuracy@5
- type: maxsim_accuracy@10
value: 0.9795918367346939
name: Maxsim Accuracy@10
- type: maxsim_precision@1
value: 0.4489795918367347
name: Maxsim Precision@1
- type: maxsim_precision@3
value: 0.45578231292517
name: Maxsim Precision@3
- type: maxsim_precision@5
value: 0.42857142857142855
name: Maxsim Precision@5
- type: maxsim_precision@10
value: 0.3979591836734694
name: Maxsim Precision@10
- type: maxsim_recall@1
value: 0.0374760765069978
name: Maxsim Recall@1
- type: maxsim_recall@3
value: 0.11222611187778031
name: Maxsim Recall@3
- type: maxsim_recall@5
value: 0.16841195565659994
name: Maxsim Recall@5
- type: maxsim_recall@10
value: 0.28231483993475376
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.43405619213135205
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.623436993845157
name: Maxsim Mrr@10
- type: maxsim_map@100
value: 0.3793154425983796
name: Maxsim Map@100
---
# ColBERT ModernBERT-base trained on MS MARCO triplets with GradCache
This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [msmarco-bm25](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
## Model Details
### Model Description
- **Model Type:** Multi-Vector Encoder
- **Base model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base)
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 128 dimensions
- **Similarity Function:** maxsim
- **Supported Modality:** Text
- **Training Dataset:**
- [msmarco-bm25](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25)
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Multi-Vector Encoder Documentation](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Multi-Vector Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=multi-vector)
### Full Model Architecture
```
MultiVectorEncoder(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
(1): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
(2): MultiVectorMask({'skiplist_words': [], 'keep_only_token_ids': None})
(3): Normalize({'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import MultiVectorEncoder
# Download from the 🤗 Hub
model = MultiVectorEncoder("tomaarsen/multivector-ModernBERT-base-msmarco-cached-contrastive")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
'what is territorial sovereignty',
]
documents = [
'territorial sovereignty. Exclusive right of a state to exercise its powers within the boundaries of its territory.',
'Territorial preservation, as Agnew explains is merely one aspect of a states territorial integrity (2005). The lack of territorial sovereignty is often a key characteristic of so-called failed states where effective monopoly over the internal means of violence is lost.',
'1 Active Transportï\x82§ Active Transport requires the cell to use energy, usually in the form of ATP.ï\x82§ Active Transport creates a charge gradient in the cell membrane. 2 For example in the mitochondrion, hydrogen ion pumps pump hydrogen ions into the intermembrane space of the organelle as part of making ATP. 3 15.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (6, 128) (24, 128)
# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[5.6691, 5.5729, 1.3079]])
```
## Evaluation
### Metrics
#### Multi Vector Information Retrieval
* Datasets: `NanoMSMARCO`, `NanoNQ`, `NanoFiQA2018`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
* Evaluated with [MultiVectorInformationRetrievalEvaluator](https://sbert.net/docs/package_reference/multi_vector_encoder/evaluation.html#sentence_transformers.multi_vector_encoder.evaluation.MultiVectorInformationRetrievalEvaluator)
| Metric | NanoMSMARCO | NanoNQ | NanoFiQA2018 | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoHotpotQA | NanoNFCorpus | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
|:--------------------|:------------|:-----------|:-------------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
| maxsim_accuracy@1 | 0.34 | 0.38 | 0.3 | 0.2 | 0.56 | 0.58 | 0.68 | 0.42 | 0.78 | 0.34 | 0.14 | 0.48 | 0.449 |
| maxsim_accuracy@3 | 0.5 | 0.5 | 0.4 | 0.38 | 0.78 | 0.9 | 0.76 | 0.5 | 0.88 | 0.54 | 0.46 | 0.6 | 0.7551 |
| maxsim_accuracy@5 | 0.6 | 0.6 | 0.44 | 0.46 | 0.82 | 0.94 | 0.78 | 0.54 | 0.9 | 0.62 | 0.62 | 0.64 | 0.8367 |
| maxsim_accuracy@10 | 0.72 | 0.7 | 0.52 | 0.54 | 0.86 | 0.98 | 0.84 | 0.56 | 0.94 | 0.76 | 0.76 | 0.7 | 0.9796 |
| maxsim_precision@1 | 0.34 | 0.38 | 0.3 | 0.2 | 0.56 | 0.58 | 0.68 | 0.42 | 0.78 | 0.34 | 0.14 | 0.48 | 0.449 |
| maxsim_precision@3 | 0.1667 | 0.1733 | 0.1667 | 0.1267 | 0.5 | 0.3067 | 0.28 | 0.3333 | 0.3667 | 0.2667 | 0.1533 | 0.2267 | 0.4558 |
| maxsim_precision@5 | 0.12 | 0.124 | 0.124 | 0.096 | 0.448 | 0.192 | 0.192 | 0.296 | 0.232 | 0.216 | 0.124 | 0.148 | 0.4286 |
| maxsim_precision@10 | 0.072 | 0.072 | 0.078 | 0.064 | 0.374 | 0.102 | 0.108 | 0.234 | 0.124 | 0.152 | 0.076 | 0.08 | 0.398 |
| maxsim_recall@1 | 0.34 | 0.36 | 0.1934 | 0.0917 | 0.0646 | 0.5467 | 0.34 | 0.0279 | 0.6807 | 0.0717 | 0.14 | 0.445 | 0.0375 |
| maxsim_recall@3 | 0.5 | 0.49 | 0.2849 | 0.1707 | 0.1224 | 0.8467 | 0.42 | 0.0967 | 0.8353 | 0.1647 | 0.46 | 0.595 | 0.1122 |
| maxsim_recall@5 | 0.6 | 0.58 | 0.3257 | 0.2063 | 0.1721 | 0.8867 | 0.48 | 0.1135 | 0.872 | 0.2217 | 0.62 | 0.64 | 0.1684 |
| maxsim_recall@10 | 0.72 | 0.65 | 0.3724 | 0.2703 | 0.2484 | 0.9333 | 0.54 | 0.136 | 0.9227 | 0.3107 | 0.76 | 0.7 | 0.2823 |
| **maxsim_ndcg@10** | **0.5197** | **0.4985** | **0.3225** | **0.2154** | **0.4666** | **0.7635** | **0.5294** | **0.3066** | **0.8471** | **0.2974** | **0.4229** | **0.5832** | **0.4341** |
| maxsim_mrr@10 | 0.4572 | 0.4622 | 0.3641 | 0.3007 | 0.6739 | 0.7285 | 0.7264 | 0.4662 | 0.8372 | 0.465 | 0.3171 | 0.5505 | 0.6234 |
| maxsim_map@100 | 0.4703 | 0.4559 | 0.285 | 0.1658 | 0.3213 | 0.6973 | 0.4468 | 0.141 | 0.8197 | 0.2265 | 0.3249 | 0.5527 | 0.3793 |
#### Multi Vector Nano BEIR
* Dataset: `NanoBEIR_mean`
* Evaluated with [MultiVectorNanoBEIREvaluator](https://sbert.net/docs/package_reference/multi_vector_encoder/evaluation.html#sentence_transformers.multi_vector_encoder.evaluation.MultiVectorNanoBEIREvaluator) with these parameters:
```json
{
"dataset_names": [
"msmarco",
"nq",
"fiqa2018"
],
"dataset_id": "sentence-transformers/NanoBEIR-en"
}
```
| Metric | Value |
|:--------------------|:-----------|
| maxsim_accuracy@1 | 0.32 |
| maxsim_accuracy@3 | 0.46 |
| maxsim_accuracy@5 | 0.5333 |
| maxsim_accuracy@10 | 0.6267 |
| maxsim_precision@1 | 0.32 |
| maxsim_precision@3 | 0.1689 |
| maxsim_precision@5 | 0.1173 |
| maxsim_precision@10 | 0.0713 |
| maxsim_recall@1 | 0.2778 |
| maxsim_recall@3 | 0.425 |
| maxsim_recall@5 | 0.4843 |
| maxsim_recall@10 | 0.5628 |
| **maxsim_ndcg@10** | **0.4339** |
| maxsim_mrr@10 | 0.4159 |
| maxsim_map@100 | 0.3931 |
#### Multi Vector Nano BEIR
* Dataset: `NanoBEIR_mean`
* Evaluated with [MultiVectorNanoBEIREvaluator](https://sbert.net/docs/package_reference/multi_vector_encoder/evaluation.html#sentence_transformers.multi_vector_encoder.evaluation.MultiVectorNanoBEIREvaluator) with these parameters:
```json
{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
],
"dataset_id": "sentence-transformers/NanoBEIR-en"
}
```
| Metric | Value |
|:--------------------|:-----------|
| maxsim_accuracy@1 | 0.4345 |
| maxsim_accuracy@3 | 0.6119 |
| maxsim_accuracy@5 | 0.6767 |
| maxsim_accuracy@10 | 0.7584 |
| maxsim_precision@1 | 0.4345 |
| maxsim_precision@3 | 0.271 |
| maxsim_precision@5 | 0.2108 |
| maxsim_precision@10 | 0.1488 |
| maxsim_recall@1 | 0.2569 |
| maxsim_recall@3 | 0.3922 |
| maxsim_recall@5 | 0.4528 |
| maxsim_recall@10 | 0.5266 |
| **maxsim_ndcg@10** | **0.4774** |
| maxsim_mrr@10 | 0.5363 |
| maxsim_map@100 | 0.4066 |
## Training Details
### Training Dataset
#### msmarco-bm25
* Dataset: [msmarco-bm25](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25) at [ce8a493](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25/tree/ce8a493a65af5e872c3c92f72a89e2e99e175f02)
* Size: 99,000 training samples
* Columns: query, positive, and negative
* Approximate statistics based on the first 100 samples:
| | query | positive | negative |
|:---------|:--------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
| type | string | string | string |
| modality | text | text | text |
| details | - min: 5 tokens
- mean: 9.4 tokens
- max: 18 tokens
| - min: 28 tokens
- mean: 83.3 tokens
- max: 197 tokens
| - min: 24 tokens
- mean: 76.0 tokens
- max: 226 tokens
|
* Samples:
| query | positive | negative |
|:------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| what is an agate made of | Agate is the name given to a group of silicate minerals that are made up primarily of chalcedony. Chalcedony is a member of the quartz family of minerals. | What is an agate, though, and what are the properties and feng shui meaning of agate? Let's find out. WHAT IS THE MEANING OF AGATE? As a form of chalcedony (type of quartz) agate exhibits a variety of colours , shapes, as well as an often present gentle iridescence. |
| what is the socratic method? | The Socratic Learning Method (SLM) is a constructivist learning approach consisting of four key. steps: eliciting relevant preconceptions, clarifying preconceptions, testing ones own. hypotheses or encountered propositions, and deciding whether to accept the hypotheses or. propositions. | The Socratic Learning Method and the Inquiry-Based Learning Method. Since the Socratic dialogues are among the earliest documented instances of learning through. inquiry, it is reasonable to argue that what is now known as inquiry-based learning can trace its. origin to the Socratic Learning Method. |
| what ditto means | ⢠DITTO (noun). The noun DITTO has 1 sense: 1. a mark used to indicate the word above it should be repeated. Familiarity information: DITTO used as a noun is very rare. ⢠DITTO (verb). The verb DITTO has 1 sense: 1. repeat an action or statement. Familiarity information: DITTO used as a verb is very rare. | Valerie Hill 0 I had texted one of my Auntie's and she sent the word Ditto back to me I was like what do that mean. Now I know!!! Acronym. DITTO in text means the same, or me too, or I agree. |
* Loss: [CachedMultiVectorMultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/multi_vector_encoder/losses.html#cachedmultivectormultiplenegativesrankingloss) with these parameters:
```json
{
"score_metric": "colbert_scores",
"mini_batch_size": 32,
"mini_batch_num_tokens": null,
"score_mini_batch_size": 32,
"scale": 1.0,
"size_average": true,
"gather_across_devices": false
}
```
### Evaluation Dataset
#### msmarco-bm25
* Dataset: [msmarco-bm25](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25) at [ce8a493](https://huggingface.co/datasets/sentence-transformers/msmarco-bm25/tree/ce8a493a65af5e872c3c92f72a89e2e99e175f02)
* Size: 1,000 evaluation samples
* Columns: query, positive, and negative
* Approximate statistics based on the first 100 samples:
| | query | positive | negative |
|:---------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string | string |
| modality | text | text | text |
| details | - min: 4 tokens
- mean: 9.18 tokens
- max: 21 tokens
| - min: 22 tokens
- mean: 76.76 tokens
- max: 210 tokens
| - min: 30 tokens
- mean: 84.36 tokens
- max: 210 tokens
|
* Samples:
| query | positive | negative |
|:------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| expected return is a function of | security market line Security market line (SML) is the representation of the capital asset pricing model. It displays the expected rate of return of an individual security as a function of systematic, non-diversifiable risk (its beta). beta Average sensitivity of a security's price to overall securities market prices. | A portfolio's expected return is the sum of the weighted average of each asset's expected return. Calculate a portfolio's expected return. To calculate the expected return of a portfolio, you need to know the expected return and weight of each asset in a portfolio. |
| what smoker temperature for barbecue chicken | Chickens smoked hot and fast over indirect heat on the grill. But for pulled chicken, I wanted a slightly more intense smokiness to balance the barbecue sauce. I decided to do a side-by-side comparison, cooking one on the grill over indirect heat at around 375°F, and one in the smoker at 225°F. | Cook for an hour and 15 minutes. Check your smoked chicken breasts to make sure the temperature is still holding at about 250 degrees. Flip the chicken breasts and close the lid again. After 30 minutes, check the internal temperature of the chicken with a meat thermometer. You are looking for a temperature of 160 degrees before you can pull them off the smoker. Serve your smoked chicken breasts with a side of barbecue sauce for dipping. |
| in classification of matter what is an element | Matter can be in the same phase or in two different phases for this separation to take place. Key Terms. mixture: Something that consists of diverse, non-bonded elements or molecules. element: A chemical substance that is made up of a particular kind of atom and cannot be broken down or transformed by a chemical reaction. | 2. What four elements make up 96% of all living matter? The four elements that make up 96% of all living matter are oxygen, carbon, hydrogen and nitrogen. 3. What is the difference between an essential element and a trace element? An essential element is an element that an organism needs to live a healthy life and reproduce. A trace element is required by an organism in only minute quantities. Section 2 4. Sketch a model of an atom of helium, showing the electrons, protons, neutrons, and atomic nucleus. Neutrons Protons Electrons 5. |
* Loss: [CachedMultiVectorMultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/multi_vector_encoder/losses.html#cachedmultivectormultiplenegativesrankingloss) with these parameters:
```json
{
"score_metric": "colbert_scores",
"mini_batch_size": 32,
"mini_batch_num_tokens": null,
"score_mini_batch_size": 32,
"scale": 1.0,
"size_average": true,
"gather_across_devices": false
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 256
- `num_train_epochs`: 1
- `learning_rate`: 3e-05
- `warmup_steps`: 0.05
- `bf16`: True
- `per_device_eval_batch_size`: 32
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
Click to expand
- `per_device_train_batch_size`: 256
- `num_train_epochs`: 1
- `max_steps`: -1
- `learning_rate`: 3e-05
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: None
- `warmup_steps`: 0.05
- `optim`: adamw_torch_fused
- `optim_args`: None
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `optim_target_modules`: None
- `gradient_accumulation_steps`: 1
- `average_tokens_across_devices`: True
- `max_grad_norm`: 1.0
- `label_smoothing_factor`: 0.0
- `bf16`: True
- `fp16`: False
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `use_cache`: False
- `neftune_noise_alpha`: None
- `torch_empty_cache_steps`: None
- `auto_find_batch_size`: False
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `include_num_input_tokens_seen`: no
- `log_level`: passive
- `log_level_replica`: warning
- `disable_tqdm`: False
- `project`: huggingface
- `trackio_space_id`: None
- `trackio_bucket_id`: None
- `trackio_static_space_id`: None
- `per_device_eval_batch_size`: 32
- `prediction_loss_only`: True
- `eval_on_start`: False
- `eval_do_concat_batches`: True
- `eval_use_gather_object`: False
- `eval_accumulation_steps`: None
- `include_for_metrics`: []
- `batch_eval_metrics`: False
- `save_only_model`: False
- `save_on_each_node`: False
- `enable_jit_checkpoint`: False
- `push_to_hub`: False
- `hub_private_repo`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_always_push`: False
- `hub_revision`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `restore_callback_states_from_checkpoint`: False
- `full_determinism`: False
- `seed`: 42
- `data_seed`: None
- `use_cpu`: False
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `dataloader_prefetch_factor`: None
- `remove_unused_columns`: True
- `label_names`: None
- `train_sampling_strategy`: random
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `ddp_static_graph`: None
- `ddp_backend`: None
- `ddp_timeout`: 1800
- `fsdp`: None
- `fsdp_config`: None
- `deepspeed`: None
- `debug`: []
- `skip_memory_metrics`: True
- `do_predict`: False
- `resume_from_checkpoint`: None
- `warmup_ratio`: None
- `local_rank`: -1
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
- `max_length`: None
Click to expand
| Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_maxsim_ndcg@10 | NanoNQ_maxsim_ndcg@10 | NanoFiQA2018_maxsim_ndcg@10 | NanoBEIR_mean_maxsim_ndcg@10 | NanoClimateFEVER_maxsim_ndcg@10 | NanoDBPedia_maxsim_ndcg@10 | NanoFEVER_maxsim_ndcg@10 | NanoHotpotQA_maxsim_ndcg@10 | NanoNFCorpus_maxsim_ndcg@10 | NanoQuoraRetrieval_maxsim_ndcg@10 | NanoSCIDOCS_maxsim_ndcg@10 | NanoArguAna_maxsim_ndcg@10 | NanoSciFact_maxsim_ndcg@10 | NanoTouche2020_maxsim_ndcg@10 |
|:---------:|:-------:|:-------------:|:---------------:|:--------------------------:|:---------------------:|:---------------------------:|:----------------------------:|:-------------------------------:|:--------------------------:|:------------------------:|:---------------------------:|:---------------------------:|:---------------------------------:|:--------------------------:|:--------------------------:|:--------------------------:|:-----------------------------:|
| -1 | -1 | - | - | 0.1572 | 0.1851 | 0.1355 | 0.1593 | - | - | - | - | - | - | - | - | - | - |
| 0.0103 | 4 | 5.9065 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0207 | 8 | 5.7869 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0310 | 12 | 5.5644 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0413 | 16 | 5.2117 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0517 | 20 | 4.7197 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0620 | 24 | 4.0890 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0724 | 28 | 3.5701 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0827 | 32 | 3.2990 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0930 | 36 | 3.0834 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1008 | 39 | - | 1.5038 | 0.3511 | 0.2964 | 0.2598 | 0.3025 | - | - | - | - | - | - | - | - | - | - |
| 0.1034 | 40 | 2.9494 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1137 | 44 | 2.7276 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1240 | 48 | 2.4742 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1344 | 52 | 2.3246 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1447 | 56 | 2.2679 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1550 | 60 | 2.1709 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1654 | 64 | 2.1077 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1757 | 68 | 2.0022 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1860 | 72 | 1.9609 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1964 | 76 | 1.9054 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2016 | 78 | - | 1.0499 | 0.4417 | 0.4054 | 0.2641 | 0.3704 | - | - | - | - | - | - | - | - | - | - |
| 0.2067 | 80 | 1.9625 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2171 | 84 | 1.9317 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2274 | 88 | 1.9280 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2377 | 92 | 1.7935 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2481 | 96 | 1.8347 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2584 | 100 | 1.8208 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2687 | 104 | 1.8619 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2791 | 108 | 1.8128 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2894 | 112 | 1.7879 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2997 | 116 | 1.7014 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3023 | 117 | - | 0.9225 | 0.4534 | 0.4303 | 0.3673 | 0.4170 | - | - | - | - | - | - | - | - | - | - |
| 0.3101 | 120 | 1.6731 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3204 | 124 | 1.6961 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3307 | 128 | 1.7301 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3411 | 132 | 1.6529 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3514 | 136 | 1.6703 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3618 | 140 | 1.6882 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3721 | 144 | 1.6177 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3824 | 148 | 1.6462 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3928 | 152 | 1.6391 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4031 | 156 | 1.5817 | 0.8342 | 0.4936 | 0.4750 | 0.3217 | 0.4301 | - | - | - | - | - | - | - | - | - | - |
| 0.4134 | 160 | 1.6026 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4238 | 164 | 1.6237 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4341 | 168 | 1.5942 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4444 | 172 | 1.6211 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4548 | 176 | 1.6076 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4651 | 180 | 1.5197 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4755 | 184 | 1.5760 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4858 | 188 | 1.5535 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4961 | 192 | 1.5019 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5039 | 195 | - | 0.8004 | 0.5052 | 0.4891 | 0.3107 | 0.4350 | - | - | - | - | - | - | - | - | - | - |
| 0.5065 | 196 | 1.5746 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5168 | 200 | 1.5618 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5271 | 204 | 1.5410 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5375 | 208 | 1.5333 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5478 | 212 | 1.5448 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5581 | 216 | 1.5221 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5685 | 220 | 1.5687 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5788 | 224 | 1.5448 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5891 | 228 | 1.4613 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5995 | 232 | 1.5090 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6047 | 234 | - | 0.7690 | 0.4905 | 0.4702 | 0.3187 | 0.4265 | - | - | - | - | - | - | - | - | - | - |
| 0.6098 | 236 | 1.4572 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6202 | 240 | 1.5143 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6305 | 244 | 1.4898 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6408 | 248 | 1.4234 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6512 | 252 | 1.3648 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6615 | 256 | 1.3490 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6718 | 260 | 1.3034 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6822 | 264 | 1.3108 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6925 | 268 | 1.2706 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7028 | 272 | 1.3064 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7054 | 273 | - | 0.6826 | 0.4836 | 0.4642 | 0.2856 | 0.4112 | - | - | - | - | - | - | - | - | - | - |
| 0.7132 | 276 | 1.2720 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7235 | 280 | 1.2861 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7339 | 284 | 1.2676 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7442 | 288 | 1.2693 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7545 | 292 | 1.3310 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7649 | 296 | 1.3174 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7752 | 300 | 1.2657 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7855 | 304 | 1.2438 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7959 | 308 | 1.2717 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8062 | 312 | 1.2444 | 0.6657 | 0.5069 | 0.4660 | 0.3159 | 0.4296 | - | - | - | - | - | - | - | - | - | - |
| 0.8165 | 316 | 1.2162 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8269 | 320 | 1.2765 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8372 | 324 | 1.2029 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8475 | 328 | 1.2716 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8579 | 332 | 1.2241 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8682 | 336 | 1.2276 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8786 | 340 | 1.2421 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8889 | 344 | 1.2431 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8992 | 348 | 1.2105 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| **0.907** | **351** | **-** | **0.6562** | **0.5197** | **0.4985** | **0.3225** | **0.4469** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
| 0.9096 | 352 | 1.2252 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9199 | 356 | 1.2255 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9302 | 360 | 1.2393 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9406 | 364 | 1.1916 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9509 | 368 | 1.1789 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9612 | 372 | 1.2024 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9716 | 376 | 1.2324 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9819 | 380 | 1.2035 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9922 | 384 | 1.2166 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 1.0 | 387 | - | 0.6495 | 0.5036 | 0.4702 | 0.3279 | 0.4339 | - | - | - | - | - | - | - | - | - | - |
| -1 | -1 | - | - | 0.5197 | 0.4985 | 0.3225 | 0.4774 | 0.2154 | 0.4666 | 0.7635 | 0.5294 | 0.3066 | 0.8471 | 0.2974 | 0.4229 | 0.5832 | 0.4341 |
* The bold row denotes the saved checkpoint.