| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext |
| pipeline_tag: fill-mask |
| --- |
| # Model Card for Model ID |
|
|
| We fine-tuned BiomedBERT using study descriptions from metagenomic projects sourced from MGnify. We applied MLM to unlabelled text data, specifically focusing on the project study descriptions. By fine-tuning the model on domain-specific text, the model now better understands the language and nuances found in metagenomics study description, which helps improve the performance of biome classification tasks. |
|
|
| This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
| - **Developed by:** SantiagoSanchezF |
| - **Model type:** MLM |
| - **Language(s) (NLP):** English |
| - **License:** [More Information Needed] |
| - **Finetuned from model:** microsoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext |
|
|
|
|
| ### Downstream Use [optional] |
|
|
| This model isthe base of SantiagoSanchezF/trapiche-biome-classifier |
|
|
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
| [More Information Needed] |
|
|
| ### Training Procedure |
|
|
| The model was domain adapted by applying masked language modeling (MLM) to a corpus of study descriptions derived from metagenomic projects in MGnify. The input text was tokenized with a maximum sequence length of 256 tokens. A data collator was configured to randomly mask 15% of the input tokens for the MLM task. Training was performed with a batch size of 8, over 3 epochs, and with a learning rate of 5e-5. |
|
|
|
|
| ## Citation [optional] |
|
|
| TBD |
|
|
|
|
