Instructions to use entropy/roberta_zinc_480m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use entropy/roberta_zinc_480m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="entropy/roberta_zinc_480m")# Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("entropy/roberta_zinc_480m") model = AutoModelForMaskedLM.from_pretrained("entropy/roberta_zinc_480m") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -94,11 +94,17 @@ The model was trained with cross entropy loss for 150,000 iterations with a batc
|
|
| 94 |
There is a [decoder model](https://huggingface.co/entropy/roberta_zinc_decoder) trained to reconstruct
|
| 95 |
inputs from embeddings generated with this model
|
| 96 |
|
| 97 |
-
#### Compression
|
| 98 |
|
| 99 |
-
There is a [compression
|
| 100 |
trained to compress embeddings generated by this model from the native size of 768 to
|
| 101 |
-
smaller sizes (512, 256, 128, 64, 32).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 102 |
|
| 103 |
|
| 104 |
**BibTeX:**
|
|
|
|
| 94 |
There is a [decoder model](https://huggingface.co/entropy/roberta_zinc_decoder) trained to reconstruct
|
| 95 |
inputs from embeddings generated with this model
|
| 96 |
|
| 97 |
+
#### Compression Encoder
|
| 98 |
|
| 99 |
+
There is a [compression encoder model](https://huggingface.co/entropy/roberta_zinc_compression_encoder)
|
| 100 |
trained to compress embeddings generated by this model from the native size of 768 to
|
| 101 |
+
smaller sizes (512, 256, 128, 64, 32) while preserving cosine similarity between embeddings.
|
| 102 |
+
|
| 103 |
+
#### Decomposer
|
| 104 |
+
|
| 105 |
+
There is a [embedding decomposer model](https://huggingface.co/entropy/roberta_zinc_enamine_decomposer)
|
| 106 |
+
trained to "decompose" a roberta-zinc embedding into two building block embeddings from the Enamine
|
| 107 |
+
library.
|
| 108 |
|
| 109 |
|
| 110 |
**BibTeX:**
|