---
tags:
- sentence-transformers
- cross-encoder
- reranker
- generated_from_trainer
- dataset_size:110400
- loss:BinaryCrossEntropyLoss
base_model: cross-encoder/ms-marco-MiniLM-L6-v2
pipeline_tag: text-ranking
library_name: sentence-transformers
---
# CrossEncoder based on cross-encoder/ms-marco-MiniLM-L6-v2
This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [cross-encoder/ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
## Model Details
### Model Description
- **Model Type:** Cross Encoder
- **Base model:** [cross-encoder/ms-marco-MiniLM-L6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2)
- **Maximum Sequence Length:** 256 tokens
- **Number of Output Labels:** 1 label
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import CrossEncoder
# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
['Mehmet Ali Ağca (Turkish pronunciation: [ˈaːdʒa]; born January 9, 1958) is a Turkish assassin of Kurdish origin who murdered left-wing journalist Abdi İpekçi on February 1, 1979, and later shot and wounded Pope John Paul II on May 13, 1981, after escaping from a Turkish prison. After serving 19 years of imprisonment in Italy where he was visited by the Pope, he was deported to Turkey, where he ser', 'christian bishop. A DBpedia category.'],
['Sucking lice (Anoplura, formerly known as Siphunculata) have around 500 species and represent the smaller of the two traditional suborders of lice. As opposed to the paraphyletic chewing lice, which are now divided among three suborders, the sucking lice are monophyletic. The Anoplura are all blood-feeding ectoparasites of mammals. They only occur on about 20% of all placentalian mammal species, a', 'eukaryote. A DBpedia category.'],
['William Ralph \\"Bill\\" Blass (June 22, 1922 – June 12, 2002) was an American fashion designer, born in Fort Wayne, Indiana. He was the recipient of many fashion awards, including seven Coty Awards and the Fashion Institute of Technology\'s Lifetime Achievement Award (1999).', 'artist. A DBpedia category.'],
['Secretariat (March 30, 1970 – October 4, 1989) was an American Thoroughbred racehorse who, in 1973, became the first Triple Crown winner in 25 years. His record-breaking win in the Belmont Stakes, where he left the field 31 lengths behind him, is widely regarded as one of the greatest races of all time. During his racing career, he won five Eclipse Awards, including Horse of the Year honors at age', 'stadium. A DBpedia category.'],
["The Hunsecker's Mill Covered Bridge is a covered bridge located in Lancaster County, Pennsylvania, United States. The bridge has a single span, wooden, double Burr arch trusses design. The bridge, which spans the Conestoga River, is 180 feet (55 m) long, making it the longest single span covered bridge in the county. The bridge's WGCB Number is 38-36-06. Unlike most historic covered bridges in the", 'baronet. A DBpedia category.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)
# Or rank different texts based on similarity to a single text
ranks = model.rank(
'Mehmet Ali Ağca (Turkish pronunciation: [ˈaːdʒa]; born January 9, 1958) is a Turkish assassin of Kurdish origin who murdered left-wing journalist Abdi İpekçi on February 1, 1979, and later shot and wounded Pope John Paul II on May 13, 1981, after escaping from a Turkish prison. After serving 19 years of imprisonment in Italy where he was visited by the Pope, he was deported to Turkey, where he ser',
[
'christian bishop. A DBpedia category.',
'eukaryote. A DBpedia category.',
'artist. A DBpedia category.',
'stadium. A DBpedia category.',
'baronet. A DBpedia category.',
]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
```
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 110,400 training samples
* Columns: sentence1, sentence2, and label
* Approximate statistics based on the first 1000 samples:
| | sentence1 | sentence2 | label |
|:--------|:------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
| type | string | string | float |
| details | - min: 35 characters
- mean: 351.7 characters
- max: 400 characters
| - min: 24 characters
- mean: 33.2 characters
- max: 69 characters
| - min: 0.0
- mean: 0.16
- max: 1.0
|
* Samples:
| sentence1 | sentence2 | label |
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------|:-----------------|
| Mehmet Ali Ağca (Turkish pronunciation: [ˈaːdʒa]; born January 9, 1958) is a Turkish assassin of Kurdish origin who murdered left-wing journalist Abdi İpekçi on February 1, 1979, and later shot and wounded Pope John Paul II on May 13, 1981, after escaping from a Turkish prison. After serving 19 years of imprisonment in Italy where he was visited by the Pope, he was deported to Turkey, where he ser | christian bishop. A DBpedia category. | 0.0 |
| Sucking lice (Anoplura, formerly known as Siphunculata) have around 500 species and represent the smaller of the two traditional suborders of lice. As opposed to the paraphyletic chewing lice, which are now divided among three suborders, the sucking lice are monophyletic. The Anoplura are all blood-feeding ectoparasites of mammals. They only occur on about 20% of all placentalian mammal species, a | eukaryote. A DBpedia category. | 0.0 |
| William Ralph \"Bill\" Blass (June 22, 1922 – June 12, 2002) was an American fashion designer, born in Fort Wayne, Indiana. He was the recipient of many fashion awards, including seven Coty Awards and the Fashion Institute of Technology's Lifetime Achievement Award (1999). | artist. A DBpedia category. | 0.0 |
* Loss: [BinaryCrossEntropyLoss](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
```json
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 32
- `learning_rate`: 2e-05
- `weight_decay`: 0.01
- `fp16`: True
#### All Hyperparameters
Click to expand
- `per_device_train_batch_size`: 32
- `num_train_epochs`: 3
- `max_steps`: -1
- `learning_rate`: 2e-05
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: None
- `warmup_steps`: 0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `weight_decay`: 0.01
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `optim_target_modules`: None
- `gradient_accumulation_steps`: 1
- `average_tokens_across_devices`: True
- `max_grad_norm`: 1.0
- `label_smoothing_factor`: 0.0
- `bf16`: False
- `fp16`: True
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `use_cache`: False
- `neftune_noise_alpha`: None
- `torch_empty_cache_steps`: None
- `auto_find_batch_size`: False
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `include_num_input_tokens_seen`: no
- `log_level`: passive
- `log_level_replica`: warning
- `disable_tqdm`: False
- `project`: huggingface
- `trackio_space_id`: trackio
- `eval_strategy`: no
- `per_device_eval_batch_size`: 8
- `prediction_loss_only`: True
- `eval_on_start`: False
- `eval_do_concat_batches`: True
- `eval_use_gather_object`: False
- `eval_accumulation_steps`: None
- `include_for_metrics`: []
- `batch_eval_metrics`: False
- `save_only_model`: False
- `save_on_each_node`: False
- `enable_jit_checkpoint`: False
- `push_to_hub`: False
- `hub_private_repo`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_always_push`: False
- `hub_revision`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `restore_callback_states_from_checkpoint`: False
- `full_determinism`: False
- `seed`: 42
- `data_seed`: None
- `use_cpu`: False
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `dataloader_prefetch_factor`: None
- `remove_unused_columns`: True
- `label_names`: None
- `train_sampling_strategy`: random
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `ddp_backend`: None
- `ddp_timeout`: 1800
- `fsdp`: []
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `deepspeed`: None
- `debug`: []
- `skip_memory_metrics`: True
- `do_predict`: False
- `resume_from_checkpoint`: None
- `warmup_ratio`: None
- `local_rank`: -1
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
Click to expand
| Epoch | Step | Training Loss |
|:------:|:-----:|:-------------:|
| 0.0145 | 50 | 0.4739 |
| 0.0290 | 100 | 0.3323 |
| 0.0435 | 150 | 0.3062 |
| 0.0580 | 200 | 0.3014 |
| 0.0725 | 250 | 0.2991 |
| 0.0870 | 300 | 0.2877 |
| 0.1014 | 350 | 0.2622 |
| 0.1159 | 400 | 0.2598 |
| 0.1304 | 450 | 0.2628 |
| 0.1449 | 500 | 0.2878 |
| 0.1594 | 550 | 0.2537 |
| 0.1739 | 600 | 0.2494 |
| 0.1884 | 650 | 0.2288 |
| 0.2029 | 700 | 0.2291 |
| 0.2174 | 750 | 0.2108 |
| 0.2319 | 800 | 0.2191 |
| 0.2464 | 850 | 0.2124 |
| 0.2609 | 900 | 0.2385 |
| 0.2754 | 950 | 0.2240 |
| 0.2899 | 1000 | 0.2017 |
| 0.3043 | 1050 | 0.2115 |
| 0.3188 | 1100 | 0.2130 |
| 0.3333 | 1150 | 0.2063 |
| 0.3478 | 1200 | 0.2098 |
| 0.3623 | 1250 | 0.2169 |
| 0.3768 | 1300 | 0.2096 |
| 0.3913 | 1350 | 0.2459 |
| 0.4058 | 1400 | 0.1833 |
| 0.4203 | 1450 | 0.1974 |
| 0.4348 | 1500 | 0.1961 |
| 0.4493 | 1550 | 0.1894 |
| 0.4638 | 1600 | 0.1935 |
| 0.4783 | 1650 | 0.1863 |
| 0.4928 | 1700 | 0.1959 |
| 0.5072 | 1750 | 0.1929 |
| 0.5217 | 1800 | 0.1886 |
| 0.5362 | 1850 | 0.1940 |
| 0.5507 | 1900 | 0.1786 |
| 0.5652 | 1950 | 0.1687 |
| 0.5797 | 2000 | 0.1944 |
| 0.5942 | 2050 | 0.1943 |
| 0.6087 | 2100 | 0.1735 |
| 0.6232 | 2150 | 0.1597 |
| 0.6377 | 2200 | 0.1954 |
| 0.6522 | 2250 | 0.1952 |
| 0.6667 | 2300 | 0.1790 |
| 0.6812 | 2350 | 0.1647 |
| 0.6957 | 2400 | 0.1792 |
| 0.7101 | 2450 | 0.1605 |
| 0.7246 | 2500 | 0.1872 |
| 0.7391 | 2550 | 0.1657 |
| 0.7536 | 2600 | 0.1871 |
| 0.7681 | 2650 | 0.1845 |
| 0.7826 | 2700 | 0.1919 |
| 0.7971 | 2750 | 0.1480 |
| 0.8116 | 2800 | 0.1701 |
| 0.8261 | 2850 | 0.1744 |
| 0.8406 | 2900 | 0.1596 |
| 0.8551 | 2950 | 0.1726 |
| 0.8696 | 3000 | 0.1710 |
| 0.8841 | 3050 | 0.1611 |
| 0.8986 | 3100 | 0.1520 |
| 0.9130 | 3150 | 0.1588 |
| 0.9275 | 3200 | 0.1614 |
| 0.9420 | 3250 | 0.1407 |
| 0.9565 | 3300 | 0.1698 |
| 0.9710 | 3350 | 0.1729 |
| 0.9855 | 3400 | 0.1493 |
| 1.0 | 3450 | 0.1587 |
| 1.0145 | 3500 | 0.1426 |
| 1.0290 | 3550 | 0.1179 |
| 1.0435 | 3600 | 0.1272 |
| 1.0580 | 3650 | 0.1422 |
| 1.0725 | 3700 | 0.1325 |
| 1.0870 | 3750 | 0.1398 |
| 1.1014 | 3800 | 0.1252 |
| 1.1159 | 3850 | 0.1164 |
| 1.1304 | 3900 | 0.1290 |
| 1.1449 | 3950 | 0.1195 |
| 1.1594 | 4000 | 0.1347 |
| 1.1739 | 4050 | 0.1665 |
| 1.1884 | 4100 | 0.1453 |
| 1.2029 | 4150 | 0.1287 |
| 1.2174 | 4200 | 0.1335 |
| 1.2319 | 4250 | 0.1410 |
| 1.2464 | 4300 | 0.1309 |
| 1.2609 | 4350 | 0.1425 |
| 1.2754 | 4400 | 0.1386 |
| 1.2899 | 4450 | 0.1334 |
| 1.3043 | 4500 | 0.1194 |
| 1.3188 | 4550 | 0.1490 |
| 1.3333 | 4600 | 0.1347 |
| 1.3478 | 4650 | 0.1531 |
| 1.3623 | 4700 | 0.1279 |
| 1.3768 | 4750 | 0.1483 |
| 1.3913 | 4800 | 0.1407 |
| 1.4058 | 4850 | 0.1222 |
| 1.4203 | 4900 | 0.1462 |
| 1.4348 | 4950 | 0.1217 |
| 1.4493 | 5000 | 0.1083 |
| 1.4638 | 5050 | 0.1377 |
| 1.4783 | 5100 | 0.1404 |
| 1.4928 | 5150 | 0.1389 |
| 1.5072 | 5200 | 0.1290 |
| 1.5217 | 5250 | 0.1354 |
| 1.5362 | 5300 | 0.1219 |
| 1.5507 | 5350 | 0.1442 |
| 1.5652 | 5400 | 0.1462 |
| 1.5797 | 5450 | 0.1440 |
| 1.5942 | 5500 | 0.1313 |
| 1.6087 | 5550 | 0.1163 |
| 1.6232 | 5600 | 0.1383 |
| 1.6377 | 5650 | 0.1357 |
| 1.6522 | 5700 | 0.1286 |
| 1.6667 | 5750 | 0.1131 |
| 1.6812 | 5800 | 0.1436 |
| 1.6957 | 5850 | 0.1233 |
| 1.7101 | 5900 | 0.1305 |
| 1.7246 | 5950 | 0.1124 |
| 1.7391 | 6000 | 0.1398 |
| 1.7536 | 6050 | 0.1325 |
| 1.7681 | 6100 | 0.1371 |
| 1.7826 | 6150 | 0.1200 |
| 1.7971 | 6200 | 0.1181 |
| 1.8116 | 6250 | 0.1107 |
| 1.8261 | 6300 | 0.1168 |
| 1.8406 | 6350 | 0.1310 |
| 1.8551 | 6400 | 0.1671 |
| 1.8696 | 6450 | 0.1257 |
| 1.8841 | 6500 | 0.1189 |
| 1.8986 | 6550 | 0.1453 |
| 1.9130 | 6600 | 0.1567 |
| 1.9275 | 6650 | 0.1047 |
| 1.9420 | 6700 | 0.1261 |
| 1.9565 | 6750 | 0.1244 |
| 1.9710 | 6800 | 0.1321 |
| 1.9855 | 6850 | 0.1275 |
| 2.0 | 6900 | 0.1103 |
| 2.0145 | 6950 | 0.1219 |
| 2.0290 | 7000 | 0.1087 |
| 2.0435 | 7050 | 0.1108 |
| 2.0580 | 7100 | 0.0927 |
| 2.0725 | 7150 | 0.1161 |
| 2.0870 | 7200 | 0.1114 |
| 2.1014 | 7250 | 0.1102 |
| 2.1159 | 7300 | 0.1106 |
| 2.1304 | 7350 | 0.1103 |
| 2.1449 | 7400 | 0.1328 |
| 2.1594 | 7450 | 0.1159 |
| 2.1739 | 7500 | 0.1107 |
| 2.1884 | 7550 | 0.1042 |
| 2.2029 | 7600 | 0.1117 |
| 2.2174 | 7650 | 0.1247 |
| 2.2319 | 7700 | 0.0953 |
| 2.2464 | 7750 | 0.1079 |
| 2.2609 | 7800 | 0.1005 |
| 2.2754 | 7850 | 0.1103 |
| 2.2899 | 7900 | 0.1095 |
| 2.3043 | 7950 | 0.1162 |
| 2.3188 | 8000 | 0.1318 |
| 2.3333 | 8050 | 0.0836 |
| 2.3478 | 8100 | 0.1214 |
| 2.3623 | 8150 | 0.1283 |
| 2.3768 | 8200 | 0.1274 |
| 2.3913 | 8250 | 0.0972 |
| 2.4058 | 8300 | 0.1121 |
| 2.4203 | 8350 | 0.1052 |
| 2.4348 | 8400 | 0.1084 |
| 2.4493 | 8450 | 0.0962 |
| 2.4638 | 8500 | 0.1065 |
| 2.4783 | 8550 | 0.0863 |
| 2.4928 | 8600 | 0.1146 |
| 2.5072 | 8650 | 0.0895 |
| 2.5217 | 8700 | 0.1141 |
| 2.5362 | 8750 | 0.1070 |
| 2.5507 | 8800 | 0.1118 |
| 2.5652 | 8850 | 0.1056 |
| 2.5797 | 8900 | 0.0831 |
| 2.5942 | 8950 | 0.1073 |
| 2.6087 | 9000 | 0.1204 |
| 2.6232 | 9050 | 0.1157 |
| 2.6377 | 9100 | 0.1119 |
| 2.6522 | 9150 | 0.0872 |
| 2.6667 | 9200 | 0.1059 |
| 2.6812 | 9250 | 0.0837 |
| 2.6957 | 9300 | 0.0872 |
| 2.7101 | 9350 | 0.0981 |
| 2.7246 | 9400 | 0.1140 |
| 2.7391 | 9450 | 0.0963 |
| 2.7536 | 9500 | 0.1141 |
| 2.7681 | 9550 | 0.1097 |
| 2.7826 | 9600 | 0.1109 |
| 2.7971 | 9650 | 0.1202 |
| 2.8116 | 9700 | 0.1067 |
| 2.8261 | 9750 | 0.1134 |
| 2.8406 | 9800 | 0.1108 |
| 2.8551 | 9850 | 0.0912 |
| 2.8696 | 9900 | 0.1160 |
| 2.8841 | 9950 | 0.1002 |
| 2.8986 | 10000 | 0.0844 |
| 2.9130 | 10050 | 0.1100 |
| 2.9275 | 10100 | 0.0930 |
| 2.9420 | 10150 | 0.1085 |
| 2.9565 | 10200 | 0.1306 |
| 2.9710 | 10250 | 0.1232 |
| 2.9855 | 10300 | 0.1019 |
| 3.0 | 10350 | 0.1212 |