consistent with clipa

README.md

@@ -1,23 +1,26 @@
 ---
+tags:
+- clip
+library_name: open_clip
+pipeline_tag: zero-shot-image-classification
 license: apache-2.0
 datasets:
-- UCSC-VLAA/Recap-DataComp-1B
+- mlfoundations/datacomp_1b
 ---
-# Model Card for ViT-L-14-CLIPS-224-Recap-DataComp-1B
+# Model card for ViT-L-14-CLIPA-datacomp1B
 
-## Model Details
-
-<!-- Provide the basic links for the model. -->
+A CLIPA-v2 model...
 
-- **Repository:** https://github.com/UCSC-VLAA/CLIPS
-- **Paper:** https://arxiv.org/abs/2411.16828
-- **Project Page:** https://ucsc-vlaa.github.io/CLIPS/
+## Model Details
+- **Model Type:** Contrastive Image-Text, Zero-Shot Image Classification.
+- **Original:** https://github.com/UCSC-VLAA/CLIPA
+- **Dataset:** mlfoundations/datacomp_1b
+- **Papers:**
+  - CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a $10,000 Budget; An Extra $4,000 Unlocks 81.8% Accuracy: https://arxiv.org/abs/2306.15658
+  - An Inverse Scaling Law for CLIP Training: https://arxiv.org/abs/2305.07017
 
 ## Model Usage
 ### With OpenCLIP
-#### Note: We made modifications to the tokenizer implementation in open_clip/tokenizer.py.
-#### For more details, refer to https://github.com/UCSC-VLAA/CLIPS.
-
 ```
 import torch
 import torch.nn.functional as F
@@ -25,8 +28,8 @@ from urllib.request import urlopen
 from PIL import Image
 from open_clip import create_model_from_pretrained, get_tokenizer
 
-model, preprocess = create_model_from_pretrained('hf-hub:UCSC-VLAA/ViT-L-14-CLIPS-224-Recap-DataComp-1B')
-tokenizer = get_tokenizer('hf-hub:UCSC-VLAA/ViT-L-14-CLIPS-224-Recap-DataComp-1B')
+model, preprocess = create_model_from_pretrained('hf-hub:ViT-L-14-CLIPA')
+tokenizer = get_tokenizer('hf-hub:ViT-L-14-CLIPA')
 
 image = Image.open(urlopen(
     'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
@@ -44,4 +47,22 @@ with torch.no_grad(), torch.cuda.amp.autocast():
     text_probs = (100.0 * image_features @ text_features.T).softmax(dim=-1)
 
 print("Label probs:", text_probs)  # prints: [[0., 0., 0., 1.0]]
-```   
+```   
+
+## Citation
+```bibtex
+@article{li2023clipav2,
+      title={CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a $10,000 Budget; An Extra $4,000 Unlocks 81.8% Accuracy}, 
+      author={Xianhang Li and Zeyu Wang and Cihang Xie},
+      journal={arXiv preprint arXiv:2306.15658},
+      year={2023},
+}
+```
+```bibtex
+@inproceedings{li2023clipa,
+      title={An Inverse Scaling Law for CLIP Training}, 
+      author={Xianhang Li and Zeyu Wang and Cihang Xie},
+      booktitle={NeurIPS},
+      year={2023},
+}
+```

added_tokens.json

@@ -0,0 +1,7 @@
+{
+  "[CLS]": 101,
+  "[MASK]": 103,
+  "[PAD]": 0,
+  "[SEP]": 102,
+  "[UNK]": 100
+}

open_clip_config.json

@@ -8,13 +8,10 @@
       "patch_size": 14,
       "no_ln_pre": true,
       "pool_type": "avg",
-      "final_ln_after_pool": true,
-      "norm_kwargs": {
-            "eps":  1e-6
-        }
+      "final_ln_after_pool": true
     },
     "text_cfg": {
-      "context_length": 80,
+      "context_length": 32,
       "vocab_size": 32000,
       "hf_tokenizer_name": "bert-base-uncased",
       "tokenizer_kwargs": {
@@ -24,13 +21,7 @@
       "heads": 12,
       "layers": 12,
       "pool_type": "last",
-      "no_causal_mask": true,
-      "act_kwargs": {
-        "approximate": "tanh"
-      },
-      "norm_kwargs": {
-            "eps":  1e-6
-        }
+      "no_causal_mask": true
     }
   },
   "preprocess_cfg": {

tokenizer_config.json

@@ -41,10 +41,10 @@
       "special": true
     }
   },
-  "clean_up_tokenization_spaces": false,
+  "additional_special_tokens": [],
+  "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
   "do_lower_case": true,
-  "extra_special_tokens": {},
   "mask_token": "[MASK]",
   "model_max_length": 512,
   "pad_token": "[PAD]",