--- library_name: transformers license: apache-2.0 base_model: allenai/specter2_base tags: - generated_from_trainer metrics: - accuracy model-index: - name: results results: [] --- # 📙 SPECTER2–WoS (Multiclass Classification on Web of Science Research Areas) This model is a fine-tuned version of [allenai/specter2_base](https://huggingface.co/allenai/specter2_base) for **multiclass** bibliometric classification using **Web of Science (WoS) Research Areas**. It achieves the following results on the evaluation set: - Loss: 1.8987 - Accuracy: 0.5763 - Precision Micro: 0.5763 - Precision Macro: 0.5744 - Recall Micro: 0.5763 - Recall Macro: 0.5741 - F1 Micro: 0.5763 - F1 Macro: 0.5691 ## Model description This model fine-tunes **SPECTER2** (`allenai/specter2_base`) to classify scientific publications into **Web of Science (WoS) Research Areas**, one of the most widely used bibliometric taxonomies. WoS Research Areas represent a 153 **broad-field disciplinary classification** assigned at the journal level. The model takes **title**, **abstract**, or **title + abstract** as input and assigns **exactly one** WoS Research Area using a **softmax** classifier. Key characteristics: - **Base model:** `allenai/specter2_base` - **Task:** multiclass document classification - **Labels:** ~150 WoS Research Areas - **Activation:** softmax - **Loss:** CrossEntropyLoss - **Output:** single best-matching Research Area This model provides high-level ## Intended uses & limitations ### Intended uses This model is designed for: - Assigning publications to **Web of Science Research Areas** - Enriching or correcting metadata in: - institutional repositories - research information systems - grant and project databases - bibliometric monitoring tools - Supporting scientometric tasks: - global discipline mapping - portfolio profiling - disciplinary trend analysis - Classifying scientific publications when only **title/abstract** is available Works with: - **title only** - **abstract only** - **title + abstract** (recommended) ### Limitations - WoS Research Areas are **journal-level labels**, not article-level annotations. → This introduces **noise**, especially for multidisciplinary journals. - Some Research Areas have **low representation**, affecting macro-F1. - Multiclass classification forces each article into **one** label, even if it spans multiple disciplines. - Not suitable for: - fine-grained WoS Categories (SCs) - ASJC Areas or MAG FoS models (use separate classifiers) - normative evaluation or ranking of research - decisions requiring verified human annotations Predictions should be interpreted as **high-level field approximations**, not exact domain attribution. ## Training and evaluation data Training data consists of documents with **Web of Science Research Area** labels, derived from **journal-level** classification. Because WoS SC/RAs are not publicly distributed, we rely on a curated dataset constructed from open-access publications where Research Areas can be inferred from the journal metadata. ### Notes on WoS Research Areas - Research Areas are high-level and **non-hierarchical**. - Articles inherit the **journal’s** assigned Research Area(s). - Some journals have **multiple RAs**, but this model uses a **single predominant label** for training. ## Training procedure ### Preprocessing - Input text constructed as: `title + ". " + abstract` - Tokenization using SPECTER2 tokenizer - Maximum sequence length: **512 tokens** ### Model - Base model: `allenai/specter2_base` - Classification head: linear → softmax - Loss: **CrossEntropyLoss** ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision Micro | Precision Macro | Recall Micro | Recall Macro | F1 Micro | F1 Macro | |:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------------:|:---------------:|:------------:|:------------:|:--------:|:--------:| | 1.8283 | 1.0 | 8540 | 1.8278 | 0.5409 | 0.5409 | 0.5243 | 0.5409 | 0.5394 | 0.5409 | 0.5174 | | 1.4971 | 2.0 | 17080 | 1.6945 | 0.5666 | 0.5666 | 0.5623 | 0.5666 | 0.5645 | 0.5666 | 0.5537 | | 1.145 | 3.0 | 25620 | 1.7103 | 0.5774 | 0.5774 | 0.5676 | 0.5774 | 0.5747 | 0.5774 | 0.5636 | | 0.8711 | 4.0 | 34160 | 1.7738 | 0.5770 | 0.5770 | 0.5753 | 0.5770 | 0.5739 | 0.5770 | 0.5681 | | 0.6207 | 5.0 | 42700 | 1.8987 | 0.5763 | 0.5763 | 0.5744 | 0.5763 | 0.5741 | 0.5763 | 0.5691 | ### Evaluation results | | precision | recall | f1-score | support | |:----------------------------------------------|------------:|----------:|-----------:|------------:| | Acoustics | 0.857143 | 0.75 | 0.8 | 40 | | Agriculture | 0.446809 | 0.355932 | 0.396226 | 59 | | Allergy | 0.823529 | 0.84 | 0.831683 | 50 | | Anatomy & Morphology | 0.864865 | 0.744186 | 0.8 | 43 | | Anesthesiology | 0.813953 | 0.625 | 0.707071 | 56 | | Anthropology | 0.419355 | 0.26 | 0.320988 | 50 | | Archaeology | 0.77551 | 0.655172 | 0.71028 | 58 | | Architecture | 0.7 | 0.538462 | 0.608696 | 52 | | Area Studies | 0.282051 | 0.244444 | 0.261905 | 45 | | Art | 0.591549 | 0.75 | 0.661417 | 56 | | Arts & Humanities - Other Topics | 0.333333 | 0.166667 | 0.222222 | 54 | | Asian Studies | 0.361702 | 0.395349 | 0.377778 | 43 | | Astronomy & Astrophysics | 0.836364 | 0.958333 | 0.893204 | 48 | | Automation & Control Systems | 0.891304 | 0.836735 | 0.863158 | 49 | | Behavioral Sciences | 0.578947 | 0.431373 | 0.494382 | 51 | | Biochemistry & Molecular Biology | 0.428571 | 0.393443 | 0.410256 | 61 | | Biodiversity & Conservation | 0.431034 | 0.595238 | 0.5 | 42 | | Biomedical Social Sciences | 0.5625 | 0.661765 | 0.608108 | 68 | | Biophysics | 0.708333 | 0.790698 | 0.747253 | 43 | | Biotechnology & Applied Microbiology | 0.636364 | 0.368421 | 0.466667 | 57 | | Business & Economics | 0.465753 | 0.73913 | 0.571429 | 46 | | Cardiovascular System & Cardiology | 0.809524 | 0.708333 | 0.755556 | 48 | | Cell Biology | 0.507463 | 0.586207 | 0.544 | 58 | | Chemistry | 0.6 | 0.612245 | 0.606061 | 49 | | Classics | 0.79661 | 0.854545 | 0.824561 | 55 | | Communication | 0.454545 | 0.480769 | 0.46729 | 52 | | Computer Science | 0.538462 | 0.528302 | 0.533333 | 53 | | Construction & Building Technology | 0.631579 | 0.6 | 0.615385 | 40 | | Criminology & Penology | 0.538462 | 0.792453 | 0.641221 | 53 | | Critical Care Medicine | 0.66 | 0.611111 | 0.634615 | 54 | | Crystallography | 1 | 0.979167 | 0.989474 | 48 | | Cultural Studies | 0.25 | 0.142857 | 0.181818 | 49 | | Dance | 0.581818 | 0.711111 | 0.64 | 45 | | Demography | 0.530612 | 0.590909 | 0.55914 | 44 | | Dentistry, Oral Surgery & Medicine | 0.574074 | 0.837838 | 0.681319 | 37 | | Dermatology | 0.545455 | 0.679245 | 0.605042 | 53 | | Developmental Biology | 0.590909 | 0.530612 | 0.55914 | 49 | | Education & Educational Research | 0.363636 | 0.666667 | 0.470588 | 36 | | Electrochemistry | 0.704918 | 0.826923 | 0.761062 | 52 | | Emergency Medicine | 0.673077 | 0.744681 | 0.707071 | 47 | | Endocrinology & Metabolism | 0.647059 | 0.634615 | 0.640777 | 52 | | Energy & Fuels | 0.486842 | 0.787234 | 0.601626 | 47 | | Engineering | 0.529412 | 0.62069 | 0.571429 | 58 | | Entomology | 0.695652 | 0.711111 | 0.703297 | 45 | | Environmental Sciences & Ecology | 0.342105 | 0.276596 | 0.305882 | 47 | | Ethnic Studies | 0.615385 | 0.507937 | 0.556522 | 63 | | Evolutionary Biology | 0.770833 | 0.74 | 0.755102 | 50 | | Family Studies | 0.538462 | 0.651163 | 0.589474 | 43 | | Film, Radio & Television | 0.571429 | 0.740741 | 0.645161 | 54 | | Fisheries | 0.722222 | 0.604651 | 0.658228 | 43 | | Food Science & Technology | 0.661765 | 0.681818 | 0.671642 | 66 | | Forestry | 0.7 | 0.648148 | 0.673077 | 54 | | Gastroenterology & Hepatology | 0.596491 | 0.755556 | 0.666667 | 45 | | General & Internal Medicine | 0.473684 | 0.195652 | 0.276923 | 46 | | Genetics & Heredity | 0.627907 | 0.586957 | 0.606742 | 46 | | Geochemistry & Geophysics | 0.8125 | 0.52 | 0.634146 | 50 | | Geography | 0.604167 | 0.557692 | 0.58 | 52 | | Geology | 0.212121 | 0.145833 | 0.17284 | 48 | | Geriatrics & Gerontology | 0.615385 | 0.603774 | 0.609524 | 53 | | Government & Law | 0.529412 | 0.580645 | 0.553846 | 62 | | Health Care Sciences & Services | 0.404255 | 0.413043 | 0.408602 | 46 | | Hematology | 0.683333 | 0.82 | 0.745455 | 50 | | History | 0.44186 | 0.431818 | 0.436782 | 44 | | History & Philosophy of Science | 0.469388 | 0.425926 | 0.446602 | 54 | | Imaging Science & Photographic Technology | 0.647059 | 0.647059 | 0.647059 | 51 | | Immunology | 0.5 | 0.52381 | 0.511628 | 63 | | Infectious Diseases | 0.5 | 0.547619 | 0.522727 | 42 | | Information Science & Library Science | 0.519481 | 0.701754 | 0.597015 | 57 | | Instruments & Instrumentation | 0.714286 | 0.581395 | 0.641026 | 43 | | Integrative & Complementary Medicine | 0.724138 | 0.7 | 0.711864 | 60 | | International Relations | 0.444444 | 0.489796 | 0.466019 | 49 | | Legal Medicine | 0.854167 | 0.82 | 0.836735 | 50 | | Life Sciences & Biomedicine - Other Topics | 0.666667 | 0.254545 | 0.368421 | 55 | | Linguistics | 0.509434 | 0.627907 | 0.5625 | 43 | | Literature | 0.431373 | 0.360656 | 0.392857 | 61 | | Marine & Freshwater Biology | 0.5 | 0.553191 | 0.525253 | 47 | | Materials Science | 0.291667 | 0.254545 | 0.271845 | 55 | | Mathematical & Computational Biology | 0.547619 | 0.469388 | 0.505495 | 49 | | Mathematical Methods In Social Sciences | 0.655738 | 0.634921 | 0.645161 | 63 | | Mathematics | 0.584906 | 0.704545 | 0.639175 | 44 | | Mechanics | 0.72093 | 0.553571 | 0.626263 | 56 | | Medical Ethics | 0.542373 | 0.653061 | 0.592593 | 49 | | Medical Informatics | 0.75 | 0.85 | 0.796875 | 60 | | Medical Laboratory Technology | 0.7 | 0.5 | 0.583333 | 42 | | Metallurgy & Metallurgical Engineering | 0.666667 | 0.680851 | 0.673684 | 47 | | Meteorology & Atmospheric Sciences | 0.76 | 0.622951 | 0.684685 | 61 | | Microbiology | 0.380282 | 0.6 | 0.465517 | 45 | | Microscopy | 0.711538 | 0.637931 | 0.672727 | 58 | | Mineralogy | 0.474359 | 0.672727 | 0.556391 | 55 | | Mining & Mineral Processing | 0.609375 | 0.764706 | 0.678261 | 51 | | Music | 0.733333 | 0.758621 | 0.745763 | 58 | | Mycology | 0.807692 | 0.84 | 0.823529 | 50 | | Neurosciences & Neurology | 0.5625 | 0.4 | 0.467532 | 45 | | Nuclear Science & Technology | 0.672727 | 0.770833 | 0.718447 | 48 | | Nursing | 0.688525 | 0.677419 | 0.682927 | 62 | | Nutrition & Dietetics | 0.567568 | 0.446809 | 0.5 | 47 | | Obstetrics & Gynecology | 0.373134 | 0.675676 | 0.480769 | 37 | | Oceanography | 0.818182 | 0.734694 | 0.774194 | 49 | | Oncology | 0.618182 | 0.548387 | 0.581197 | 62 | | Operations Research & Management Science | 0.606061 | 0.689655 | 0.645161 | 58 | | Ophthalmology | 0.803279 | 0.844828 | 0.823529 | 58 | | Optics | 0.666667 | 0.625 | 0.645161 | 48 | | Orthopedics | 0.462687 | 0.837838 | 0.596154 | 37 | | Otorhinolaryngology | 0.656716 | 0.709677 | 0.682171 | 62 | | Paleontology | 0.639344 | 0.684211 | 0.661017 | 57 | | Parasitology | 0.482759 | 0.538462 | 0.509091 | 52 | | Pathology | 0.384615 | 0.454545 | 0.416667 | 44 | | Pediatrics | 0.5 | 0.638298 | 0.560748 | 47 | | Pharmacology & Pharmacy | 0.487179 | 0.422222 | 0.452381 | 45 | | Philosophy | 0.488889 | 0.511628 | 0.5 | 43 | | Physical Geography | 0.651163 | 0.622222 | 0.636364 | 45 | | Physical Sciences - Other Topics | 0.452381 | 0.44186 | 0.447059 | 43 | | Physics | 0.714286 | 0.555556 | 0.625 | 54 | | Physiology | 0.5625 | 0.5 | 0.529412 | 54 | | Plant Sciences | 0.660377 | 0.686275 | 0.673077 | 51 | | Polymer Science | 0.677966 | 0.784314 | 0.727273 | 51 | | Psychiatry | 0.603774 | 0.744186 | 0.666667 | 43 | | Psychology | 0.477273 | 0.381818 | 0.424242 | 55 | | Public Administration | 0.476923 | 0.62 | 0.53913 | 50 | | Public, Environmental & Occupational Health | 0 | 0 | 0 | 4 | | Radiology, Nuclear Medicine & Medical Imaging | 0.645833 | 0.645833 | 0.645833 | 48 | | Rehabilitation | 0.6 | 0.679245 | 0.637168 | 53 | | Religion | 0.491525 | 0.604167 | 0.542056 | 48 | | Remote Sensing | 0.5625 | 0.586957 | 0.574468 | 46 | | Reproductive Biology | 0.736842 | 0.571429 | 0.643678 | 49 | | Research & Experimental Medicine | 0.32 | 0.145455 | 0.2 | 55 | | Respiratory System | 0.55814 | 0.648649 | 0.6 | 37 | | Rheumatology | 0.729167 | 0.76087 | 0.744681 | 46 | | Robotics | 0.555556 | 0.731707 | 0.631579 | 41 | | Science & Technology - Other Topics | 0.142857 | 0.106383 | 0.121951 | 47 | | Social Issues | 0.6 | 0.176471 | 0.272727 | 51 | | Social Sciences - Other Topics | 0.153846 | 0.0677966 | 0.0941176 | 59 | | Social Work | 0.392157 | 0.5 | 0.43956 | 40 | | Sociology | 0.263158 | 0.0980392 | 0.142857 | 51 | | Spectroscopy | 0.673469 | 0.717391 | 0.694737 | 46 | | Sport Sciences | 0.714286 | 0.731707 | 0.722892 | 41 | | Substance Abuse | 0.76 | 0.791667 | 0.77551 | 48 | | Surgery | 0.363636 | 0.355556 | 0.359551 | 45 | | Technology - Other Topics | 0.214286 | 0.0576923 | 0.0909091 | 52 | | Telecommunications | 0.784314 | 0.727273 | 0.754717 | 55 | | Theater | 0.648148 | 0.614035 | 0.630631 | 57 | | Thermodynamics | 0.636364 | 0.446809 | 0.525 | 47 | | Toxicology | 0.545455 | 0.782609 | 0.642857 | 46 | | Transplantation | 0.853659 | 0.897436 | 0.875 | 39 | | Transportation | 0.58209 | 0.75 | 0.655462 | 52 | | Tropical Medicine | 0.4 | 0.425532 | 0.412371 | 47 | | Urban Studies | 0.615385 | 0.489796 | 0.545455 | 49 | | Urology & Nephrology | 0.622222 | 0.666667 | 0.643678 | 42 | | Veterinary Sciences | 0.532258 | 0.647059 | 0.584071 | 51 | | Virology | 0.710526 | 0.658537 | 0.683544 | 41 | | Water Resources | 0.592593 | 0.615385 | 0.603774 | 52 | | Women's Studies | 0.404255 | 0.413043 | 0.408602 | 46 | | Zoology | 0.452381 | 0.487179 | 0.469136 | 39 | | accuracy | 0.585616 | 0.585616 | 0.585616 | 0.585616 | | macro avg | 0.578248 | 0.584159 | 0.572667 | 7592 | | weighted avg | 0.583763 | 0.585616 | 0.576288 | 7592 | ### Framework versions - Transformers 4.57.1 - Pytorch 2.8.0+cu126 - Datasets 3.6.0 - Tokenizers 0.22.1