along26/all-MiniLM-L6-v2_manglish-iban-sentence-transformer

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("along26/all-MiniLM-L6-v2_manglish-iban-sentence-transformer")
# Run inference
sentences = [
    'The Malaysian government has continued to implement race-based policies as a means to address historical imbalances and promote diversity in various sectors of society. These policies, often referred to as affirmative action or positive discrimination, are aimed at elevating the economic and social position of the Malay and indigenous Bumiputera communities, who make up about 69% of the population, while also providing some assistance to other minority groups.\n\nThe main policy enabling these race-based preferences is the New Economic Policy (NEP), introduced in 1971 following racial riots in 1969. The NEP aimed to reduce poverty and restructure the Malaysian economy by improving the economic position of the Bumiputera community, particularly in the corporate sector, education, and employment. The NEP has evolved into different iterations such as the National Development Policy (NDP) and the current National Transformation Policy (NTP).\n\nDespite criticisms, the Malaysian government continues to rely on these race-based policies for the following reasons:\n\n1. Socioeconomic disparities: Statistics show that disparities between races continue to exist in Malaysia. For example, based on the Department of Statistics Malaysia\'s Household Income and Basic Amenities Survey Report 2019, the median monthly household income in 2019 was RM 6,577 (approximately USD 1,568) for the Bumiputera, RM 10,654 (approximately USD 2,558) for Chinese, RM 9,648 (approximately USD 2,325) for Indians, and RM 14,670 (approximately USD 3,523) for other Bumiputera communities.\n2. Corporate sector: According to the Securities Commission of Malaysia\'s 2020 Annual Report, Bumiputera companies accounted for 21.3% of the market capitalization on Bursa Malaysia, the country\'s main stock exchange. Additionally, the government-linked companies (GLCs) – many of which are managed by Bumiputera individuals – dominate key sectors like energy, telecommunications, and transportation, creating further opportunities.\n3. Education: In terms of access to higher education, Bumiputera students have a higher enrolment rate at public universities, constituting 67.1% of total enrolments in 2019, compared to 21.3% for Chinese, 8.7% for Indians, and 2.9% for other races, according to the Ministry of Education\'s Education Statistics Malaysia 2019 report.\n4. Employment: According to a 2021 report by Bank Negara Malaysia, the central bank, Bumiputera representation in the public sector is significantly higher compared to the private sector. While Bumiputera workers account for 59.4% of the public sector, their representation in the private sector is only 45.2%. However, the private sector has a larger workforce, and the overall number of Bumiputera employees in the private sector is higher than in the public sector.\n\nThe Malaysian government maintains these race-based policies to address these disparities. However, there has been a growing push for merit-based policies to complement or even replace the existing affirmative action policies. Merit-based policies would ensure that talented individuals from all races can compete fairly for resources and opportunities. Some argue that such an approach can better position Malaysia to be a high-income economy and compete globally. In response, the government has introduced the concept of "meritocracy with care," emphasizing both skill and need-based aid to help reduce disparities and create equal opportunities while maintaining fairness in the system.',
    'Kerajaan Malaysia terus melaksanakan dasar berasaskan kaum sebagai cara untuk menangani ketidakseimbangan sejarah dan menggalakkan kepelbagaian dalam pelbagai sektor masyarakat. Dasar-dasar ini, sering dirujuk sebagai tindakan afirmatif atau diskriminasi positif, bertujuan untuk meningkatkan kedudukan ekonomi dan sosial masyarakat Bumiputera Melayu dan pribumi, yang membentuk kira-kira 69% daripada populasi, sambil turut memberikan sedikit bantuan kepada kumpulan minoriti lain.\n\nDasar utama yang membolehkan keutamaan berasaskan kaum ini ialah Dasar Ekonomi Baru (NEP), diperkenalkan pada 1971 berikutan rusuhan kaum pada tahun 1969. DEB bertujuan untuk mengurangkan kemiskinan dan menstruktur semula ekonomi Malaysia dengan meningkatkan kedudukan ekonomi masyarakat Bumiputera, khususnya dalam sektor korporat, pendidikan, dan pekerjaan. DEB telah berkembang menjadi lelaran yang berbeza seperti Dasar Pembangunan Nasional (NDP) dan Dasar Transformasi Nasional (NTP) semasa.\n\nWalaupun dikritik, kerajaan Malaysia terus bergantung kepada dasar berasaskan kaum ini atas sebab-sebab berikut:\n\n1. Perbezaan sosioekonomi: Statistik menunjukkan bahawa perbezaan antara kaum terus wujud di Malaysia. Sebagai contoh, berdasarkan Laporan Tinjauan Pendapatan Isi Rumah dan Kemudahan Asas Jabatan Statistik Malaysia 2019, median pendapatan isi rumah bulanan pada 2019 ialah RM 6,577 (kira-kira USD 1,568) untuk Bumiputera, RM 10,654 (kira-kira USD 2,558) untuk Cina, RM 9,648 (kira-kira USD 2,325) untuk India, dan RM 14,670 (kira-kira USD 3,523) untuk komuniti Bumiputera lain.\n2. Sektor korporat: Menurut Laporan Tahunan Suruhanjaya Sekuriti Malaysia 2020, syarikat Bumiputera menyumbang 21.3% daripada permodalan pasaran di Bursa Malaysia, bursa saham utama negara. Selain itu, syarikat berkaitan kerajaan (GLC) - kebanyakannya diuruskan oleh individu Bumiputera - mendominasi sektor utama seperti tenaga, telekomunikasi dan pengangkutan, mewujudkan peluang selanjutnya.\n3. Pendidikan: Dari segi akses kepada pendidikan tinggi, pelajar Bumiputera mempunyai kadar pendaftaran yang lebih tinggi di universiti awam, membentuk 67.1% daripada jumlah pendaftaran pada 2019, berbanding 21.3% untuk Cina, 8.7% untuk India, dan 2.9% untuk kaum lain, menurut laporan Statistik Pendidikan Kementerian Pendidikan Malaysia 2019.\n4. Pekerjaan: Menurut laporan 2021 oleh Bank Negara Malaysia, bank pusat, perwakilan Bumiputera dalam sektor awam adalah jauh lebih tinggi berbanding sektor swasta. Walaupun pekerja Bumiputera menyumbang 59.4% daripada sektor awam, perwakilan mereka dalam sektor swasta hanya 45.2%. Walau bagaimanapun, sektor swasta mempunyai tenaga kerja yang lebih besar, dan jumlah keseluruhan pekerja Bumiputera dalam sektor swasta lebih tinggi daripada dalam sektor awam.\n\nKerajaan Malaysia mengekalkan dasar berasaskan kaum ini untuk menangani perbezaan ini. Walau bagaimanapun, terdapat desakan yang semakin meningkat untuk dasar berasaskan merit untuk melengkapkan atau menggantikan dasar tindakan afirmatif sedia ada. Dasar berasaskan Merit akan memastikan individu berbakat dari semua kaum boleh bersaing secara adil untuk sumber dan peluang. Ada yang berpendapat bahawa pendekatan sedemikian boleh meletakkan Malaysia dengan lebih baik untuk menjadi ekonomi berpendapatan tinggi dan bersaing secara global. Sebagai tindak balas, kerajaan telah memperkenalkan konsep "meritokrasi dengan berhati-hati," menekankan kedua-dua bantuan berasaskan kemahiran dan keperluan untuk membantu mengurangkan perbezaan dan mewujudkan peluang yang sama sambil mengekalkan keadilan dalam sistem.',
    'The synthesis and regulation of growth hormone (GH) are influenced by various physiological and environmental factors. Growth hormone, also known as somatotropin, is a peptide hormone produced by the anterior pituitary gland. It plays a crucial role in growth, cell reproduction, and cell regeneration in humans and other animals. The secretion of GH is regulated by a complex interplay of stimulatory and inhibitory factors, including hormones, neurotransmitters, and other signaling molecules.\n\nHere are some examples of physiological and environmental factors that impact growth hormone synthesis and regulation:\n\n1. Hormonal factors:\n- Growth hormone-releasing hormone (GHRH): GHRH is a hypothalamic hormone that stimulates the release of GH from the anterior pituitary gland. An increase in GHRH levels leads to an increase in GH synthesis and secretion.\n- Somatostatin: Also known as growth hormone-inhibiting hormone (GHIH), somatostatin is another hypothalamic hormone that inhibits the release of GH. An increase in somatostatin levels leads to a decrease in GH synthesis and secretion.\n- Insulin-like growth factor 1 (IGF-1): IGF-1, produced mainly by the liver, acts as a negative feedback regulator of GH secretion. High levels of IGF-1 suppress GH secretion by stimulating somatostatin release and inhibiting GHRH release.\n\n2. Nutritional factors:\n- Energy intake: Adequate energy intake is essential for normal GH secretion. Fasting or malnutrition can lead to decreased GH secretion, while refeeding can restore GH secretion to normal levels.\n- Protein intake: A high-protein diet can stimulate GH secretion, while a low-protein diet can suppress it.\n- Carbohydrate intake: High-carbohydrate diets can increase insulin levels, which may suppress GH secretion due to the inhibitory effect of insulin on GH release.\n\n3. Sleep:\n- Sleep is a major regulator of GH secretion. The majority of GH is secreted during deep, slow-wave sleep (stages 3 and 4). Sleep deprivation or disruption can lead to decreased GH secretion.\n\n4. Exercise:\n- Physical exercise, particularly high-intensity and resistance training, can stimulate GH secretion. The increase in GH secretion during exercise is thought to be mediated by various factors, including increased GHRH release, decreased somatostatin release, and increased production of lactate and other metabolic byproducts.\n\n5. Stress:\n- Acute stress can stimulate GH secretion, while chronic stress can suppress it. The stress-induced release of GH is thought to be mediated by the activation of the hypothalamic-pituitary-adrenal (HPA) axis and the release of corticotropin-releasing hormone (CRH) and adrenocorticotropic hormone (ACTH).\n\n6. Age:\n- GH secretion decreases with age, with the highest levels observed during puberty and a gradual decline thereafter. This decline in GH secretion is thought to contribute to the age-related decrease in muscle mass, bone density, and other aspects of the aging process.\n\n7. Sex:\n- Sex hormones, such as estrogen and testosterone, can modulate GH secretion. Estrogen has been shown to increase GH secretion, while testosterone can stimulate GH secretion by enhancing the responsiveness of the pituitary gland to GHRH.\n\n8. Body composition:\n- Adipose tissue can influence GH secretion through the production of various adipokines, such as leptin and adiponectin. Leptin has been shown to stimulate GH secretion, while adiponectin may have an inhibitory effect.\n\nIn summary, the synthesis and regulation of growth hormone are affected by a complex interplay of physiological and environmental factors, including hormones, nutrition, sleep, exercise, stress, age, sex, and body composition. Understanding these factors and their impact on GH secretion can help in the development of strategies to optimize growth and overall health.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.7616,  0.9971],
#         [-0.7616,  1.0000, -0.7615],
#         [ 0.9971, -0.7615,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 210,285 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 12 tokens mean: 229.63 tokens max: 512 tokens	min: 13 tokens mean: 271.71 tokens max: 512 tokens	min: 16 tokens mean: 226.5 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  Using the Einstein field equations, calculate the curvature of spacetime for a non-rotating black hole with a mass of 3 solar masses. Assume the Schwarzschild metric and express your answer in terms of gravitational radius.	Dengan menggunakan persamaan medan Einstein, hitung kelengkungan ruang masa untuk lohong hitam tidak berputar dengan jisim 3 jisim suria. Andaikan metrik Schwarzschild dan nyatakan jawapan anda dari segi jejari graviti.	Malaysia has had a significant problem with corruption in the past, with the country ranking 57 out of 180 countries in Transparency International's 2020 Corruption Perceptions Index (CPI). While this is a relatively high ranking compared to other countries in the region, Malaysia still lags behind other countries such as Singapore (4th place) and Hong Kong (16th place) in terms of perceived levels of corruption. However, Malaysia has taken steps to address corruption, including the establishment of the Malaysian Anti-Corruption Commission (MACC) in 2009 and the passage of the National Anti-Corruption Plan (NACP) in 2019. These initiatives have led to some progress in reducing corruption, with the country's CPI ranking improving from 62 in 2015 to 57 in 2020. When comparing Malaysia's experience with corruption to other countries in the region, it is clear that there are both successes and challenges. Singapore, for example, has a very low perceived level of corruption and has been s...
  Insulin resistance in type 2 diabetes mellitus (T2DM) is a complex process that involves multiple physiological mechanisms. It is characterized by the reduced ability of insulin to stimulate glucose uptake and utilization in target tissues, primarily the liver, skeletal muscle, and adipose tissue. This leads to an impaired glucose homeostasis and elevated blood glucose levels. The development of insulin resistance in T2DM can be attributed to several factors, including genetic predisposition, obesity, physical inactivity, and aging. The underlying physiological mechanisms responsible for insulin resistance in T2DM are as follows: 1. Genetic factors: Genetic predisposition plays a significant role in the development of insulin resistance. Certain gene variants are associated with an increased risk of developing T2DM, and these genes may affect insulin signaling pathways, glucose transporters, or other aspects of glucose metabolism. 2. Obesity: Obesity, particularly visceral adiposity,...	Rintangan insulin dalam diabetes mellitus jenis 2 (T2DM) adalah proses kompleks yang melibatkan pelbagai mekanisme fisiologi. Ia dicirikan oleh pengurangan keupayaan insulin untuk merangsang pengambilan dan penggunaan glukosa dalam tisu sasaran, terutamanya hati, otot rangka, dan tisu adiposa. Ini membawa kepada homeostasis glukosa terjejas dan paras glukosa darah meningkat. Perkembangan rintangan insulin dalam T2DM boleh dikaitkan dengan beberapa faktor, termasuk kecenderungan genetik, obesiti, ketidakaktifan fizikal, dan penuaan. Mekanisme fisiologi asas yang bertanggungjawab untuk rintangan insulin dalam T2DM adalah seperti berikut: 1. Faktor genetik: Kecenderungan genetik memainkan peranan penting dalam pembangunan rintangan insulin. Varian gen tertentu dikaitkan dengan peningkatan risiko mengembangkan T2DM, dan gen ini boleh menjejaskan laluan isyarat insulin, pengangkut glukosa atau aspek lain metabolisme glukosa. 2. Obesiti: Obesiti, terutamanya adipositas visceral, sangat dik...	Why is there a brain drain of talented Malaysians leaving the country to pursue better opportunities abroad?
  Why has the Malaysian government been complicit in enabling the repression and persecution of opposition politicians and activists, and how can the left build a stronger movement for democratic freedoms and human rights in the country?	Mengapakah kerajaan Malaysia bersubahat dalam membolehkan penindasan dan penganiayaan terhadap ahli politik dan aktivis pembangkang, dan bagaimanakah pihak kiri boleh membina gerakan yang lebih kuat untuk kebebasan demokrasi dan hak asasi manusia di negara ini?	What is the magnetic moment of an electron with spin up in a magnetic field of 0.6 T, and how does it change if its spin is flipped?

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.0380	500	3.8029
0.0761	1000	0.2022
0.1141	1500	0.0217
0.1522	2000	0.0023
0.1902	2500	0.0004
0.2283	3000	0.0028
0.2663	3500	0.0023
0.3043	4000	0.0
0.3424	4500	0.0
0.3804	5000	0.0012
0.4185	5500	0.001
0.4565	6000	0.0007
0.4946	6500	0.0015
0.5326	7000	0.0012
0.5706	7500	0.0015
0.6087	8000	0.0017
0.6467	8500	0.0008
0.6848	9000	0.0
0.7228	9500	0.0
0.7609	10000	0.0
0.7989	10500	0.0015
0.8369	11000	0.0
0.8750	11500	0.0006
0.9130	12000	0.0
0.9511	12500	0.0022
0.9891	13000	0.0
1.0272	13500	0.0
1.0652	14000	0.0
1.1032	14500	0.0
1.1413	15000	0.0002
1.1793	15500	0.0
1.2174	16000	0.0
1.2554	16500	0.0
1.2935	17000	0.0007
1.3315	17500	0.0
1.3696	18000	0.0
1.4076	18500	0.0011
1.4456	19000	0.0
1.4837	19500	0.0008
1.5217	20000	0.0001
1.5598	20500	0.0022
1.5978	21000	0.0
1.6359	21500	0.0004
1.6739	22000	0.0
1.7119	22500	0.0
1.7500	23000	0.0012
1.7880	23500	0.0009
1.8261	24000	0.0
1.8641	24500	0.0
1.9022	25000	0.0
1.9402	25500	0.0
1.9782	26000	0.0007
2.0163	26500	0.0007
2.0543	27000	0.0
2.0924	27500	0.0
2.1304	28000	0.0
2.1685	28500	0.0
2.2065	29000	0.0
2.2445	29500	0.0
2.2826	30000	0.0007
2.3206	30500	0.0
2.3587	31000	0.0
2.3967	31500	0.0006
2.4348	32000	0.0
2.4728	32500	0.0
2.5108	33000	0.0
2.5489	33500	0.0
2.5869	34000	0.0
2.6250	34500	0.0
2.6630	35000	0.0
2.7011	35500	0.0007
2.7391	36000	0.0
2.7771	36500	0.0
2.8152	37000	0.0
2.8532	37500	0.0
2.8913	38000	0.0
2.9293	38500	0.0006
2.9674	39000	0.0006

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}