CrossEncoder based on BAAI/bge-reranker-v2-m3

This is a Cross Encoder model finetuned from BAAI/bge-reranker-v2-m3 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

Model Type: Cross Encoder
Base model: BAAI/bge-reranker-v2-m3
Maximum Sequence Length: 8192 tokens
Number of Output Labels: 1 label
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Documentation: Cross Encoder Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'XLMRobertaForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of inputs
pairs = [
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product [here](https://tepo.app/products/conditional-logic) <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our [guide](https://support.tepo.app/en/article/set-up-conditional-logic-1sp9tac/). If you prefer watching than reading, feel free to take a look at our tutorial video [here](https://www.youtube.com/watch?v=7voAaJ2lzkc)"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions [here](https://support.tepo.app/en/article/change-product-images-based-on-selected-options-1831rew/) to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide [here](https://support.tepo.app/en/article/customize-style-for-app-options-using-css-26kmin/) for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', 'Good day~ This is Robin from Tepo Support team! I hope your day has been amazing so far! To make sure I’m on the same page, when you say getting the older version of the option set, are you referring to undoing some changes after you’ve already hit save? If I’ve misunderstood anything, please don’t hesitate to correct me. I want to make sure I assist you as best as I can!'],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "> Hello there, > This is Hazel from Tepo Support team. I hope your day is going great! > Thanks for sharing your concern with us. Regarding the packing slip, we do have a step-by-step instruction [here](https://support.tepo.app/en/article/how-to-show-options-in-shopify-packing-slip-1dyay7y/) that you can follow more easily on your end. Feel free to give it a try on your end to show Tepo options in your packing slip In Shopify, a courier slip is also known as a packing slip. I've seen that Hazel has sent you the instruction to add the code into the packing slip template to show the TEPO options already. May I ask if you give it a try on your end yet?"],
]
scores = model.predict(pairs)
print(scores)
# [9.4035e-01 1.8822e-03 2.3369e-05 2.3369e-05 2.2474e-05]

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Hi, is there a way to show an image once the personalization toggle is on?',
    [
        "Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product [here](https://tepo.app/products/conditional-logic) <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our [guide](https://support.tepo.app/en/article/set-up-conditional-logic-1sp9tac/). If you prefer watching than reading, feel free to take a look at our tutorial video [here](https://www.youtube.com/watch?v=7voAaJ2lzkc)",
        "Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions [here](https://support.tepo.app/en/article/change-product-images-based-on-selected-options-1831rew/) to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!",
        "Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide [here](https://support.tepo.app/en/article/customize-style-for-app-options-using-css-26kmin/) for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!",
        'Good day~ This is Robin from Tepo Support team! I hope your day has been amazing so far! To make sure I’m on the same page, when you say getting the older version of the option set, are you referring to undoing some changes after you’ve already hit save? If I’ve misunderstood anything, please don’t hesitate to correct me. I want to make sure I assist you as best as I can!',
        "> Hello there, > This is Hazel from Tepo Support team. I hope your day is going great! > Thanks for sharing your concern with us. Regarding the packing slip, we do have a step-by-step instruction [here](https://support.tepo.app/en/article/how-to-show-options-in-shopify-packing-slip-1dyay7y/) that you can follow more easily on your end. Feel free to give it a try on your end to show Tepo options in your packing slip In Shopify, a courier slip is also known as a packing slip. I've seen that Hazel has sent you the instruction to add the code into the packing slip template to show the TEPO options already. May I ask if you give it a try on your end yet?",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

Datasets: train-evaluator and test_evaluator

Evaluated with CrossEncoderRerankingEvaluator with these parameters:

{
    "at_k": 10,
    "always_rerank_positives": false
}

Metric	train-evaluator	test_evaluator
map	0.9935 (-0.0065)	0.8883 (-0.1117)
mrr@10	0.9935 (-0.0065)	0.8883 (-0.1117)
ndcg@10	0.9952 (-0.0048)	0.9166 (-0.0834)

Training Details

Training Dataset

Unnamed Dataset

Size: 5,400 training samples
Columns: sentence1, sentence2, and label
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 label
type string string float
details
min: 9 tokens
mean: 55.62 tokens
max: 487 tokens

min: 36 tokens
mean: 169.9 tokens
max: 968 tokens

min: 0.0
mean: 0.17
max: 1.0

	sentence1	sentence2	label
type	string	string	float
details	min: 9 tokens mean: 55.62 tokens max: 487 tokens	min: 36 tokens mean: 169.9 tokens max: 968 tokens	min: 0.0 mean: 0.17 max: 1.0

Samples:

sentence1	sentence2	label
`Hi, is there a way to show an image once the personalization toggle is on?`	`Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product here <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our guide. If you prefer watching than reading, feel free to take a look at our tutorial video here`	`1.0`
`Hi, is there a way to show an image once the personalization toggle is on?`	`Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions here to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!`	`0.0`
`Hi, is there a way to show an image once the personalization toggle is on?`	`Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide here for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!`	`0.0`

Loss: BinaryCrossEntropyLoss with these parameters:

{
    "activation_fn": "torch.nn.modules.linear.Identity",
    "pos_weight": 5
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 1
gradient_accumulation_steps: 32
learning_rate: 2e-05
weight_decay: 0.2
max_grad_norm: 0.5
num_train_epochs: 5
warmup_steps: 0.25
seed: 67
fp16: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 1
per_device_eval_batch_size: 8
gradient_accumulation_steps: 32
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.2
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 0.5
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0.25
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 67
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: True
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	train-evaluator_ndcg@10	test_evaluator_ndcg@10
-1	-1	-	0.8634 (-0.1366)	0.8746 (-0.1254)
0.0059	1	1.4427	-	-
0.0296	5	1.0900	-	-
0.0593	10	1.8380	-	-
0.0889	15	1.1262	-	-
0.1185	20	0.9075	-	-
0.1481	25	0.8731	-	-
0.1778	30	0.7740	-	-
0.2074	35	1.1410	-	-
0.2370	40	0.7601	-	-
0.2667	45	0.7899	-	-
0.2963	50	0.6283	0.8800 (-0.1200)	0.8871 (-0.1129)
0.3259	55	0.7696	-	-
0.3556	60	0.9953	-	-
0.3852	65	0.7479	-	-
0.4148	70	0.6350	-	-
0.4444	75	0.8330	-	-
0.4741	80	0.6033	-	-
0.5037	85	0.8338	-	-
0.5333	90	0.9694	-	-
0.5630	95	0.9039	-	-
0.5926	100	0.8650	0.9112 (-0.0888)	0.8927 (-0.1073)
0.6222	105	0.6094	-	-
0.6519	110	0.6067	-	-
0.6815	115	0.5759	-	-
0.7111	120	0.8194	-	-
0.7407	125	0.7709	-	-
0.7704	130	0.7898	-	-
0.8	135	0.8466	-	-
0.8296	140	0.9746	-	-
0.8593	145	0.8148	-	-
0.8889	150	0.5513	0.9312 (-0.0688)	0.9096 (-0.0904)
0.9185	155	0.5055	-	-
0.9481	160	0.5476	-	-
0.9778	165	0.7248	-	-
1.0059	170	0.6541	-	-
1.0356	175	0.5039	-	-
1.0652	180	0.4411	-	-
1.0948	185	0.6942	-	-
1.1244	190	0.5658	-	-
1.1541	195	0.6088	-	-
1.1837	200	0.4049	0.9347 (-0.0653)	0.9030 (-0.0970)
1.2133	205	0.6112	-	-
1.2430	210	0.2897	-	-
1.2726	215	0.4576	-	-
1.3022	220	0.7817	-	-
1.3319	225	0.7286	-	-
1.3615	230	0.5475	-	-
1.3911	235	0.4749	-	-
1.4207	240	0.5976	-	-
1.4504	245	0.4889	-	-
1.48	250	0.8006	0.9478 (-0.0522)	0.8905 (-0.1095)
1.5096	255	0.5885	-	-
1.5393	260	0.5553	-	-
1.5689	265	0.6979	-	-
1.5985	270	0.4054	-	-
1.6281	275	0.4447	-	-
1.6578	280	0.6768	-	-
1.6874	285	0.6879	-	-
1.7170	290	0.4486	-	-
1.7467	295	0.8144	-	-
1.7763	300	0.6580	0.9526 (-0.0474)	0.9174 (-0.0826)
1.8059	305	0.5297	-	-
1.8356	310	0.5700	-	-
1.8652	315	0.3300	-	-
1.8948	320	0.8602	-	-
1.9244	325	0.6331	-	-
1.9541	330	0.6205	-	-
1.9837	335	0.5258	-	-
2.0119	340	0.6980	-	-
2.0415	345	0.2407	-	-
2.0711	350	0.3267	0.9633 (-0.0367)	0.9107 (-0.0893)
2.1007	355	0.2947	-	-
2.1304	360	0.1790	-	-
2.16	365	0.3841	-	-
2.1896	370	0.3123	-	-
2.2193	375	0.4142	-	-
2.2489	380	0.6400	-	-
2.2785	385	0.2441	-	-
2.3081	390	0.2816	-	-
2.3378	395	0.4786	-	-
2.3674	400	0.5176	0.9684 (-0.0316)	0.9196 (-0.0804)
2.3970	405	0.3519	-	-
2.4267	410	0.3983	-	-
2.4563	415	0.5621	-	-
2.4859	420	0.3907	-	-
2.5156	425	0.4945	-	-
2.5452	430	0.3195	-	-
2.5748	435	0.5087	-	-
2.6044	440	0.5812	-	-
2.6341	445	0.3276	-	-
2.6637	450	0.3166	0.9796 (-0.0204)	0.9174 (-0.0826)
2.6933	455	0.4656	-	-
2.7230	460	0.4002	-	-
2.7526	465	0.3733	-	-
2.7822	470	0.3798	-	-
2.8119	475	0.3161	-	-
2.8415	480	0.4545	-	-
2.8711	485	0.4441	-	-
2.9007	490	0.2575	-	-
2.9304	495	0.2315	-	-
2.96	500	0.4730	0.9826 (-0.0174)	0.9200 (-0.0800)
2.9896	505	0.4888	-	-
3.0178	510	0.2182	-	-
3.0474	515	0.1584	-	-
3.0770	520	0.1502	-	-
3.1067	525	0.1653	-	-
3.1363	530	0.1713	-	-
3.1659	535	0.2844	-	-
3.1956	540	0.1836	-	-
3.2252	545	0.0927	-	-
3.2548	550	0.3476	0.9832 (-0.0168)	0.9213 (-0.0787)
3.2844	555	0.3332	-	-
3.3141	560	0.3291	-	-
3.3437	565	0.2665	-	-
3.3733	570	0.1910	-	-
3.4030	575	0.5073	-	-
3.4326	580	0.2244	-	-
3.4622	585	0.1066	-	-
3.4919	590	0.2117	-	-
3.5215	595	0.1337	-	-
3.5511	600	0.1933	0.9886 (-0.0114)	0.9118 (-0.0882)
3.5807	605	0.0638	-	-
3.6104	610	0.1539	-	-
3.64	615	0.1943	-	-
3.6696	620	0.2148	-	-
3.6993	625	0.2873	-	-
3.7289	630	0.6843	-	-
3.7585	635	0.2633	-	-
3.7881	640	0.5072	-	-
3.8178	645	0.2672	-	-
3.8474	650	0.1996	0.9912 (-0.0088)	0.9210 (-0.0790)
3.8770	655	0.1643	-	-
3.9067	660	0.1246	-	-
3.9363	665	0.5416	-	-
3.9659	670	0.3488	-	-
3.9956	675	0.1317	-	-
4.0237	680	0.2893	-	-
4.0533	685	0.0652	-	-
4.0830	690	0.1084	-	-
4.1126	695	0.0768	-	-
4.1422	700	0.0968	0.9946 (-0.0054)	0.9223 (-0.0777)
4.1719	705	0.2517	-	-
4.2015	710	0.1219	-	-
4.2311	715	0.1915	-	-
4.2607	720	0.0576	-	-
4.2904	725	0.2852	-	-
4.32	730	0.1978	-	-
4.3496	735	0.1791	-	-
4.3793	740	0.1776	-	-
4.4089	745	0.0889	-	-
4.4385	750	0.1076	0.9958 (-0.0042)	0.9186 (-0.0814)
4.4681	755	0.0545	-	-
4.4978	760	0.0901	-	-
4.5274	765	0.1187	-	-
4.5570	770	0.1037	-	-
4.5867	775	0.0583	-	-
4.6163	780	0.1538	-	-
4.6459	785	0.1026	-	-
4.6756	790	0.0472	-	-
4.7052	795	0.0937	-	-
4.7348	800	0.1225	0.9952 (-0.0048)	0.9166 (-0.0834)
4.7644	805	0.2453	-	-
4.7941	810	0.0514	-	-
4.8237	815	0.1391	-	-
4.8533	820	0.0984	-	-
4.8830	825	0.2722	-	-
4.9126	830	0.1562	-	-
4.9422	835	0.0289	-	-
4.9719	840	0.0779	-	-
5.0	845	0.1853	-	-

The bold row denotes the saved checkpoint.

Training Time

Training: 1.5 hours
Evaluation: 23.9 minutes
Total: 1.9 hours

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.4.0
Transformers: 5.0.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

Downloads last month: 1

Safetensors

Model size

0.6B params

Tensor type

F32

Inference Providers NEW

Text Ranking

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for CuongCao/oe-bge-reranker-v2-m3-ft-v5

Base model

BAAI/bge-reranker-v2-m3

Finetuned

(83)

this model

Paper for CuongCao/oe-bge-reranker-v2-m3-ft-v5

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Paper • 1908.10084 • Published Aug 27, 2019 • 14

Evaluation results

Map on train evaluator
self-reported

0.994
Mrr@10 on train evaluator
self-reported

0.994
Ndcg@10 on train evaluator
self-reported

0.995
Map on test evaluator
self-reported

0.888
Mrr@10 on test evaluator
self-reported

0.888
Ndcg@10 on test evaluator
self-reported

0.917