CuongCao/oe-bge-reranker-large-ft-v2

CrossEncoder based on BAAI/bge-reranker-large

This is a Cross Encoder model finetuned from BAAI/bge-reranker-large using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: BAAI/bge-reranker-large
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product [here](https://tepo.app/products/conditional-logic) <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our [guide](https://support.tepo.app/en/article/set-up-conditional-logic-1sp9tac/). If you prefer watching than reading, feel free to take a look at our tutorial video [here](https://www.youtube.com/watch?v=7voAaJ2lzkc)"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions [here](https://support.tepo.app/en/article/change-product-images-based-on-selected-options-1831rew/) to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide [here](https://support.tepo.app/en/article/customize-style-for-app-options-using-css-26kmin/) for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!"],
    ['Hi, is there a way to show an image once the personalization toggle is on?', 'Good day~ This is Robin from Tepo Support team! I hope your day has been amazing so far! To make sure I’m on the same page, when you say getting the older version of the option set, are you referring to undoing some changes after you’ve already hit save? If I’ve misunderstood anything, please don’t hesitate to correct me. I want to make sure I assist you as best as I can!'],
    ['Hi, is there a way to show an image once the personalization toggle is on?', "> Hello there, > This is Hazel from Tepo Support team. I hope your day is going great! > Thanks for sharing your concern with us. Regarding the packing slip, we do have a step-by-step instruction [here](https://support.tepo.app/en/article/how-to-show-options-in-shopify-packing-slip-1dyay7y/) that you can follow more easily on your end. Feel free to give it a try on your end to show Tepo options in your packing slip In Shopify, a courier slip is also known as a packing slip. I've seen that Hazel has sent you the instruction to add the code into the packing slip template to show the TEPO options already. May I ask if you give it a try on your end yet?"],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Hi, is there a way to show an image once the personalization toggle is on?',
    [
        "Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product [here](https://tepo.app/products/conditional-logic) <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our [guide](https://support.tepo.app/en/article/set-up-conditional-logic-1sp9tac/). If you prefer watching than reading, feel free to take a look at our tutorial video [here](https://www.youtube.com/watch?v=7voAaJ2lzkc)",
        "Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions [here](https://support.tepo.app/en/article/change-product-images-based-on-selected-options-1831rew/) to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!",
        "Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide [here](https://support.tepo.app/en/article/customize-style-for-app-options-using-css-26kmin/) for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!",
        'Good day~ This is Robin from Tepo Support team! I hope your day has been amazing so far! To make sure I’m on the same page, when you say getting the older version of the option set, are you referring to undoing some changes after you’ve already hit save? If I’ve misunderstood anything, please don’t hesitate to correct me. I want to make sure I assist you as best as I can!',
        "> Hello there, > This is Hazel from Tepo Support team. I hope your day is going great! > Thanks for sharing your concern with us. Regarding the packing slip, we do have a step-by-step instruction [here](https://support.tepo.app/en/article/how-to-show-options-in-shopify-packing-slip-1dyay7y/) that you can follow more easily on your end. Feel free to give it a try on your end to show Tepo options in your packing slip In Shopify, a courier slip is also known as a packing slip. I've seen that Hazel has sent you the instruction to add the code into the packing slip template to show the TEPO options already. May I ask if you give it a try on your end yet?",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Dataset: support-rerank-eval
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": false
  }
  

Metric	Value
map	0.9087 (-0.0913)
mrr@10	0.9087 (-0.0913)
ndcg@10	0.9315 (-0.0685)

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,400 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	float
  details	min: 28 characters mean: 218.29 characters max: 1961 characters	min: 141 characters mean: 643.39 characters max: 3722 characters	min: 0.0 mean: 0.17 max: 1.0

• Samples:

  sentence1	sentence2	label
  Hi, is there a way to show an image once the personalization toggle is on?	Hi there, This is Hazel from Tepo Support team. I hope your day is going great! Yes, you can ultilize our conditional settings to show an image based on the toggle's selection, as demonstrated on our demo product here <3 For step-by-step instructions on how to set up conditional settings on your end, please check out our guide. If you prefer watching than reading, feel free to take a look at our tutorial video here	1.0
  Hi, is there a way to show an image once the personalization toggle is on?	Hello there, I apologize for getting back to you a bit later, as we've been away during non-working hours. I hope your weekend went great! For your information, we have a feature to change the product image based on the option's selection. I have enabled this feature for your store, so feel free to follow our step-by-step instructions here to set up this feature for your store. Please let me know if you encounter any difficulties. I am more than happy to help!	0.0
  Hi, is there a way to show an image once the personalization toggle is on?	Hello, Thank you for contacting Tepo Support team! I hope you're having a wonderful day! To enlarge the size of the image swatches, you can refer to our guide here for step-by-step instructions. This will allow you to make the necessary adjustments on your end. Feel free to give it a try, and if you encounter any difficulties or if coding isn't your forte, please don't hesitate to share the product URL with me. I'll be more than happy to assist further!	0.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 1
• gradient_accumulation_steps: 32
• learning_rate: 1e-05
• num_train_epochs: 5
• warmup_ratio: 0.2
• warmup_steps: 0.2
• seed: 67
• fp16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 1
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 32
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.2
• warmup_steps: 0.2
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 67
• data_seed: None
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: True
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Image

Epoch	Step	Training Loss	support-rerank-eval_ndcg@10
-1	-1	-	0.8413 (-0.1587)
0.0119	1	1.3864	-
0.0593	5	2.2658	-
0.1185	10	1.5262	-
0.1778	15	1.5737	-
0.2370	20	1.3171	-
0.2963	25	1.0806	-
0.3556	30	1.0587	-
0.4148	35	0.8149	-
0.4741	40	0.8706	-
0.5333	45	1.1330	-
0.5926	50	1.1210	-
0.6519	55	0.7105	-
0.7111	60	0.8596	-
0.7704	65	0.8416	-
0.8296	70	0.6543	-
0.8889	75	0.7784	-
0.9481	80	0.5754	-
1.0	85	0.7647	-
1.0593	90	0.5856	-
1.1185	95	0.5812	-
1.1778	100	0.5459	0.8906 (-0.1094)
1.2370	105	0.5259	-
1.2963	110	0.6609	-
1.3556	115	0.5932	-
1.4148	120	0.5482	-
1.4741	125	0.7539	-
1.5333	130	0.6486	-
1.5926	135	0.6475	-
1.6519	140	0.5627	-
1.7111	145	0.6848	-
1.7704	150	0.6890	-
1.8296	155	0.5712	-
1.8889	160	0.5949	-
1.9481	165	0.7066	-
2.0	170	0.7013	-
2.0593	175	0.3729	-
2.1185	180	0.4327	-
2.1778	185	0.3703	-
2.2370	190	0.4843	-
2.2963	195	0.3460	-
2.3556	200	0.3679	0.9241 (-0.0759)
2.4148	205	0.4412	-
2.4741	210	0.5300	-
2.5333	215	0.4922	-
2.5926	220	0.4407	-
2.6519	225	0.3835	-
2.7111	230	0.4581	-
2.7704	235	0.4687	-
2.8296	240	0.3968	-
2.8889	245	0.3373	-
2.9481	250	0.3882	-
3.0	255	0.5394	-
3.0593	260	0.3229	-
3.1185	265	0.2746	-
3.1778	270	0.4415	-
3.2370	275	0.2434	-
3.2963	280	0.3033	-
3.3556	285	0.3377	-
3.4148	290	0.3316	-
3.4741	295	0.2425	-
3.5333	300	0.3449	0.9141 (-0.0859)
3.5926	305	0.2569	-
3.6519	310	0.3693	-
3.7111	315	0.2660	-
3.7704	320	0.3553	-
3.8296	325	0.3099	-
3.8889	330	0.2923	-
3.9481	335	0.3726	-
4.0	340	0.5681	-
4.0593	345	0.1964	-
4.1185	350	0.2471	-
4.1778	355	0.3227	-
4.2370	360	0.2911	-
4.2963	365	0.2578	-
4.3556	370	0.2402	-
4.4148	375	0.2367	-
4.4741	380	0.1998	-
4.5333	385	0.2359	-
4.5926	390	0.2177	-
4.6519	395	0.1999	-
4.7111	400	0.2293	0.9315 (-0.0685)
4.7704	405	0.2748	-
4.8296	410	0.2312	-
4.8889	415	0.3013	-
4.9481	420	0.1602	-
5.0	425	0.3283	-

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.3
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.8.3
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}