Sentence Similarity
sentence-transformers
Safetensors
Bengali
metaclip_2
trimmed
lbourdois commited on
Commit
93733dd
·
verified ·
1 Parent(s): a85673d

Trimmed MetaCLIP-2 text vocab for Bengali

Browse files
Files changed (1) hide show
  1. README.md +6 -34
README.md CHANGED
@@ -14,41 +14,13 @@ datasets:
14
 
15
  # metaclip-2-worldwide-m16-384-ben-32768
16
 
17
- This model is a 82.6% smaller version of [facebook/metaclip-2-worldwide-m16-384](https://huggingface.co/facebook/metaclip-2-worldwide-m16-384)
18
- optimized for Bengali language via **text vocabulary trimming** mined on
19
- [Lumberjackk/fineweb-2-trimming](https://huggingface.co/datasets/Lumberjackk/fineweb-2-trimming).
20
-
21
- Only the **text encoder** vocabulary and embeddings are trimmed.
22
- The **vision encoder** is kept identical to the original.
23
 
24
  ## Model Statistics
25
- - **Original text vocabulary size:** 901,629 tokens
26
- - **Trimmed text vocabulary size:** 32,767 tokens
27
  - **Vocabulary reduction:** 96.4%
28
- - **Original model size:** 538,547,201 parameters
29
- - **Trimmed model size:** 93,690,369 parameters
30
  - **Size reduction:** 82.6%
31
-
32
- ## Usage
33
-
34
- ```python
35
- from transformers import AutoProcessor, AutoModel
36
- import torch
37
-
38
- model = AutoModel.from_pretrained("provisoirement/metaclip-2-worldwide-m16-384-ben-32768")
39
- processor = AutoProcessor.from_pretrained("provisoirement/metaclip-2-worldwide-m16-384-ben-32768")
40
-
41
- # Text-only encoding
42
- texts = ["a photo of a cat", "a photo of a dog"]
43
- inputs = processor(text=texts, return_tensors="pt", padding=True)
44
- with torch.inference_mode():
45
- text_features = model.get_text_features(**inputs)
46
-
47
- # Image + text (standard CLIP usage)
48
- # from PIL import Image
49
- # image = Image.open("image.jpg")
50
- # inputs = processor(images=image, text=texts, return_tensors="pt", padding=True)
51
- # with torch.inference_mode():
52
- # outputs = model(**inputs)
53
- # logits_per_image = outputs.logits_per_image
54
- ```
 
14
 
15
  # metaclip-2-worldwide-m16-384-ben-32768
16
 
17
+ This model is a 82.6% smaller version of [facebook/metaclip-2-worldwide-m16-384](https://huggingface.co/facebook/metaclip-2-worldwide-m16-384)
18
+ optimized for Bengali via text vocabulary trimming.
 
 
 
 
19
 
20
  ## Model Statistics
21
+ - **Original vocab size:** 901,629
22
+ - **Trimmed vocab size:** 32,768
23
  - **Vocabulary reduction:** 96.4%
24
+ - **Original params:** 538,547,201
25
+ - **Trimmed params:** 93,690,369
26
  - **Size reduction:** 82.6%