---
license: mit
---
## SARS-Cov-2-Pro

The **SARS-CoV-2-Pro** is a protein model trained on a masked language modeling objective, resulting from the unsupervised fine-tuning of the **ESM-2 Long** (https://huggingface.co/gabrielbianchin/esm2_t33_long) protein language model. Its fine-tuning dataset is sourced from SARS-Cov-2 S protein sequences in the NCBI Virus database (as of February 20, 2025). The original dataset comprises **2,584,107** protein sequences from different species, reduced to **134,555** sequences after removing duplicates and those with 100% identity.

The **SARS-CoV-2-Pro** model is suitable for downstream applications related to SARS-Cov-2 viruses. For detailed information on the model’s training data, please refer to the accompanying paper.

Multiple **SARS-CoV-2-Pro** checkpoints are available on the Hub, with varying model sizes. Generally, larger models offer higher accuracy but require more memory and training time.

| Checkpoint name | Num layers | Num parameters |
|------------------------------|----|----------|
| .[SARS_Cov_2_Pro_t33_650M_esm2](https://huggingface.co/sihuapeng/SARS_Cov_2_Pro_t33_esm2_Long) | 33 | 650M      | 
| [SARS_Cov_2_Pro_t30_150M_esm2](https://huggingface.co/sihuapeng/SARS_Cov_2_Pro_t30_esm2_Long) | 30 | 150M    | 
| [SARS_Cov_2_Pro_t12_35M_esm2](https://huggingface.co/sihuapeng/SARS_Cov_2_Pro_t12_esm2_Long) | 12 | 35M     | 
| [SARS_Cov_2_Pro_t6_8M_esm2](https://huggingface.co/sihuapeng/SARS_Cov_2_Pro_t6_esm2_Long)  | 6  | 8M      | 

note: final clean data.