Khrawsynth commited on
Commit
d5031eb
·
verified ·
1 Parent(s): 36a159c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -3
README.md CHANGED
@@ -38,7 +38,7 @@ base_model: sentence-transformers/LaBSE
38
 
39
  ## What is NE-Embed?
40
 
41
- NE-Embed is the first text embedding model purpose-built for Northeast Indian languages. It enables **semantic search, retrieval, and RAG** across languages that general multilingual models fail on — languages like Garo, Meitei, Nyishi, and Pnar where raw LaBSE achieves under 15% retrieval accuracy.
42
 
43
  Trained on 201,738 balanced parallel pairs across 10 languages using bi-encoder fine-tuning with `MultipleNegativesRankingLoss`.
44
 
@@ -89,7 +89,7 @@ Evaluated on 500 samples per language. CLRI = Cross-Language Retrieval Interfere
89
  ```python
90
  from sentence_transformers import SentenceTransformer
91
 
92
- model = SentenceTransformer("MWirelabs/ne-embed-v1")
93
 
94
  sentences = [
95
  "Where is the nearest hospital?", # English
@@ -117,7 +117,17 @@ score = 0.7 * ne_embed_score + 0.3 * bm25_score
117
  - **Loss:** `MultipleNegativesRankingLoss`
118
  - **Data:** 201,738 English↔NE language parallel pairs, capped at 25k per language to prevent Assamese attractor bias
119
  - **Epochs:** 3 · **Batch size:** 64 · **Max seq length:** 128
120
- - **Hardware:** NVIDIA A40 · **Training time:** ~1.3 hours
 
 
 
 
 
 
 
 
 
 
121
 
122
  ---
123
 
 
38
 
39
  ## What is NE-Embed?
40
 
41
+ NE-Embed A multilingual text embedding model purpose-built for Northeast Indian languages. It enables **semantic search, retrieval, and RAG** across languages that general multilingual models fail on — languages like Garo, Meitei, Nyishi, and Pnar where raw LaBSE achieves under 15% retrieval accuracy.
42
 
43
  Trained on 201,738 balanced parallel pairs across 10 languages using bi-encoder fine-tuning with `MultipleNegativesRankingLoss`.
44
 
 
89
  ```python
90
  from sentence_transformers import SentenceTransformer
91
 
92
+ model = SentenceTransformer("MWirelabs/ne-embed")
93
 
94
  sentences = [
95
  "Where is the nearest hospital?", # English
 
117
  - **Loss:** `MultipleNegativesRankingLoss`
118
  - **Data:** 201,738 English↔NE language parallel pairs, capped at 25k per language to prevent Assamese attractor bias
119
  - **Epochs:** 3 · **Batch size:** 64 · **Max seq length:** 128
120
+ - **Hardware:** NVIDIA A40 (48 GB) · **Training time:** ~1.3 hours
121
+
122
+ ---
123
+
124
+ ## Intended Uses
125
+
126
+ - Semantic search
127
+ - Dense retrieval
128
+ - RAG
129
+ - Cross-lingual retrieval
130
+ - Clustering
131
 
132
  ---
133