new stuff in readme.md

readme.md

@@ -36,9 +36,14 @@ However, this kind of text, without more information, is not useful to learn a g
   
   Example: .... 
 
-+ MSCOCO-IT.
++ MSCOCO-IT. This image-caption dataset comes from the work by Antonio et al., 2019. The captions comes from the original 
+MSCOCO dataset and have been translated with Microsoft Translator. The 2017 version of the MSCOCO training set contains more than
+100K images, for each image more than one caption is available.
 
-+ Conceptual Captions.
++ Conceptual Captions. This image-caption dataset comes from the work by Sharma et al., 2018. There are more than 3mln image-caption pairs in
+this dataset and these have been collected from the web. We downloaded the images with the URLs provided by the dataset, but we
+could not retrieve them all. Eventually, we had to translate the captions to Italian. We have been able to collect
+a dataset with 700K translated captions.
 
 
 ## Better Augmentations
@@ -66,6 +71,10 @@ To better understand how well our clip-italian model works we run an experimenta
 
 ### mCLIP
 
+The multilingual CLIP (henceforth, mCLIP), is a model introduced by [Nils Reimers](https://www.sbert.net/docs/pretrained_models.html) in his
+[sentence-transformer](https://www.sbert.net/index.html) library. mCLIP is based on a multilingual encoder
+that was created through multilingual knowledge distillation (see Reimers et al., 2020).
+
 ### Experiments Replication
 We provide two colab notebooks to replicate both experiments.
 
@@ -77,15 +86,25 @@ We selected two different tasks:
 
 ### Image Retrieval
 
+This experiment is run against the MSCOCO-IT validation set (that we haven't used in training). Given in input
+a caption, we search for the most similar image in the MSCOCO-IT validation set. As evaluation metrics
+we use the MRR.
+
 | MRR             | CLIP-Italian | mCLIP |
 | --------------- | ------------ |-------|
 | MRR@1           | **0.3797**   | 0.2874|   
 | MRR@5           | **0.5039**   | 0.3957|
 | MRR@10          | **0.5204**   | 0.4129|
 
+Is it true that we used MSCOCO-IT in training, and this might give us an advantage. However the original CLIP model was trained
+on 400million images (and some of them probably were from MSCOCO).
+
 [Colab: Image Retrieval Evaluation](https://colab.research.google.com/drive/1bLVwVKpAndpEDHqjzxVPr_9nGrSbuOQd?usp=sharing)
 
-### Zero-shot classification
+### Zero-shot image classification
+
+This experiment replicates the original one run by OpenAI on zero-shot image classification. 
+
 
 | Accuracy        | CLIP-Italian | mCLIP |
 | --------------- | ------------ |-------|
@@ -96,11 +115,28 @@ We selected two different tasks:
 
 [Colab: ImageNet Zero Shot Evaluation](https://colab.research.google.com/drive/1zfWeVWY79XXH63Ci-pk8xxx3Vu_RRgW-?usp=sharing)
 
+Our results confirm that CLIP-Italian is very competitive and beats mCLIP on the two different task
+we have been testing. Note, however, that our results are lower than those shown in the original OpenAI
+paper (see, Radford et al., 2021), considering that our results are in line with those obtained by mCLIP we think that 
+the translated image labels might have had an impact on the final scores.
+
 ## Qualitative Evaluation
 
+### Colors
+
+### Numbers
+
 # Broader Outlook
 
+# References
+
+Antonio, S., Croce, D., & Basili, R. (2019). Large scale datasets for Image and Video Captioning in Italian. IJCoL. Italian Journal of Computational Linguistics, 5(5-2), 49-60.
+
+Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018, July). Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2556-2565).
+
+Reimers, N., & Gurevych, I. (2020, November). Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4512-4525).
 
+Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning Transferable Visual Models From Natural Language Supervision. ICML.
 
 # Other Notes
 This readme has been designed using resources from Flaticon.com