Hugging Face's logo

johnyy212
/
moe-girl-v3

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:58482
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on BAAI/bge-small-zh-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-zh-v1.5 on the json dataset. It maps sentences & paragraphs to a 512-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-small-zh-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 512 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text
  • Training Dataset:
    • json

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 512, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("johnyy212/moe-girl-v3")
# Run inference
sentences = [
    '御姐、长短袜、下乳、长靴、红瞳',
    '角色：少女前线:PTRD\n本名：PTRD41反坦克步枪\n别名：卤蛋、南半球\n声优：清水爱\n萌点：长发、巨乳、黑丝、御姐、下乳、呆毛、哥萨克帽\n发色：亚麻\n瞳色：红',
    '少女前线:PTRD',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.5500, -0.1585],
#         [ 0.5500,  1.0000, -0.1041],
#         [-0.1585, -0.1041,  1.0000]])

Training Details

Training Dataset

json

  • Dataset: json
  • Size: 58,482 training samples
  • Columns: anchor, positive, and char_name
  • Approximate statistics based on the first 1000 samples:
    anchor positive char_name
    type string string string
    details
    • min: 3 tokens
    • mean: 11.96 tokens
    • max: 36 tokens
    • min: 13 tokens
    • mean: 77.78 tokens
    • max: 478 tokens
    • min: 3 tokens
    • mean: 7.23 tokens
    • max: 23 tokens
  • Samples:
    anchor positive char_name
    杀手、吐槽、红发、混血儿 角色：风吹雪
    本名：风吹雪
    别名：雪妹、花、天意之花
    年龄：16
    瞳色：棕
    发色：红
    血型：O
    萌点：长直、傲娇、女神系、吐槽、妹妹、裹胸布    		 风吹雪
    黑发、百合、高中生、半马尾、黑色连裤袜 角色：高远凛
    本名：高远、凛
    别名：凛、Rin
    声优：长妻树里
    生日：10月10日
    瞳色：棕
    发色：黑
    三围：85、60、86
    萌点：高中生、短发、穿越者、百合、黑丝、半马尾    		 高远凛
    小野凉子、北见六花 角色：世计丸
    本名：世计、丸
    声优：北见六花配音角色、小野凉子配音角色
    发色：紫
    瞳色：紫
    三围：90、58、81
    生日：12月1日
    星座：射手
    萌点：过膝袜、长发、下双马尾、毒舌、绝对领域、姐控、遮眼发、巨乳、傲娇    		 世计丸
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Evaluation Dataset

json

  • Dataset: json
  • Size: 3,615 evaluation samples
  • Columns: anchor, positive, and char_name
  • Approximate statistics based on the first 1000 samples:
    anchor positive char_name
    type string string string
    details
    • min: 9 tokens
    • mean: 15.88 tokens
    • max: 29 tokens
    • min: 21 tokens
    • mean: 77.17 tokens
    • max: 307 tokens
    • min: 3 tokens
    • mean: 7.31 tokens
    • max: 24 tokens
  • Samples:
    anchor positive char_name
    萝莉控、黑色吊带袜、手套、绿瞳 角色：碧蓝航线:皇家方舟
    本名：HMS Ark Royal
    别名：狱友
    发色：紫
    瞳色：绿
    萌点：御姐、黑色吊带袜、萝莉控、巨乳、遮眼发
    生日：4月13日    		 碧蓝航线:皇家方舟
    孤儿、紫瞳、披风、长直、黑发、黑长直 角色：江昙
    本名：江昙、Jiang Tan
    别名：“圣女”、小昙花
    声优：白杺瓒、皛四白、富田美忧
    发色：黑
    瞳色：紫
    萌点：披风、针织衫、黑长直、孤儿    		 江昙
    尾巴、自大、嗜杀 角色：紫影狒狒
    本名：Purprill、the、Mandroid、紫影狒狒
    声优：高木涉
    瞳色：绿
    萌点：狒狒、尾巴、自大、嗜杀    		 紫影狒狒
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 64
  • learning_rate: 2e-05
  • num_train_epochs: 5
  • warmup_steps: 0.1
  • fp16: True
  • load_best_model_at_end: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • do_predict: False
  • prediction_loss_only: True
  • per_device_train_batch_size: 64
  • per_device_eval_batch_size: 8
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 5
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_ratio: None
  • warmup_steps: 0.1
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • enable_jit_checkpoint: False
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • use_cpu: False
  • seed: 42
  • data_seed: None
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: -1
  • ddp_backend: None
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • auto_find_batch_size: False
  • full_determinism: False
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • use_cache: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss Validation Loss
0.1094 100 2.4634 -
0.2188 200 1.1738 -
0.3282 300 1.0204 -
0.4376 400 0.8706 -
0.5470 500 0.8237 -
0.6565 600 0.8114 -
0.7659 700 0.7423 -
0.8753 800 0.7134 -
0.9847 900 0.6652 -
1.0 914 - 0.1627
1.0941 1000 0.6096 -
1.2035 1100 0.6270 -
1.3129 1200 0.6147 -
1.4223 1300 0.5881 -
1.5317 1400 0.6169 -
1.6411 1500 0.5780 -
1.7505 1600 0.5719 -
1.8600 1700 0.5977 -
1.9694 1800 0.5682 -
2.0 1828 - 0.1404
2.0788 1900 0.5367 -
2.1882 2000 0.5234 -
2.2976 2100 0.4955 -
2.4070 2200 0.5051 -
2.5164 2300 0.5314 -
2.6258 2400 0.4933 -
2.7352 2500 0.4881 -
2.8446 2600 0.4982 -
2.9540 2700 0.4992 -
3.0 2742 - 0.1312
3.0635 2800 0.4945 -
3.1729 2900 0.4468 -
3.2823 3000 0.4524 -
3.3917 3100 0.4675 -
3.5011 3200 0.4591 -
3.6105 3300 0.4627 -
3.7199 3400 0.4545 -
3.8293 3500 0.4718 -
3.9387 3600 0.4558 -
4.0 3656 - 0.1237
4.0481 3700 0.4439 -
4.1575 3800 0.4375 -
4.2670 3900 0.4384 -
4.3764 4000 0.4486 -
4.4858 4100 0.4373 -
4.5952 4200 0.4356 -
4.7046 4300 0.4425 -
4.8140 4400 0.4285 -
4.9234 4500 0.4219 -
5.0 4570 - 0.1243
  • The bold row denotes the saved checkpoint.

Training Time

  • Training: 15.3 minutes

Framework Versions

  • Python: 3.12.13
  • Sentence Transformers: 5.4.1
  • Transformers: 5.0.0
  • PyTorch: 2.10.0+cu128
  • Accelerate: 1.13.0
  • Datasets: 4.8.5
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
118
Safetensors
Model size
24M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for johnyy212/moe-girl-v3

Base model

BAAI/bge-small-zh-v1.5
Finetuned
(9)
this model

Papers for johnyy212/moe-girl-v3