SIRIS-Lab/specter2-wos-multiclass

📙 SPECTER2–WoS (Multiclass Classification on Web of Science Research Areas)

This model is a fine-tuned version of allenai/specter2_base for multiclass bibliometric classification using Web of Science (WoS) Research Areas.

It achieves the following results on the evaluation set:

• Loss: 1.8987
• Accuracy: 0.5763
• Precision Micro: 0.5763
• Precision Macro: 0.5744
• Recall Micro: 0.5763
• Recall Macro: 0.5741
• F1 Micro: 0.5763
• F1 Macro: 0.5691

Model description

This model fine-tunes SPECTER2 (allenai/specter2_base) to classify scientific publications into Web of Science (WoS) Research Areas, one of the most widely used bibliometric taxonomies.

WoS Research Areas represent a 153 broad-field disciplinary classification assigned at the journal level. The model takes title, abstract, or title + abstract as input and assigns exactly one WoS Research Area using a softmax classifier.

Key characteristics:

• Base model: allenai/specter2_base
• Task: multiclass document classification
• Labels: ~150 WoS Research Areas
• Activation: softmax
• Loss: CrossEntropyLoss
• Output: single best-matching Research Area

This model provides high-level

Intended uses & limitations

Intended uses

This model is designed for:

• Assigning publications to Web of Science Research Areas
• Enriching or correcting metadata in:
  • institutional repositories
  • research information systems
  • grant and project databases
  • bibliometric monitoring tools
• Supporting scientometric tasks:
  • global discipline mapping
  • portfolio profiling
  • disciplinary trend analysis
• Classifying scientific publications when only title/abstract is available

Works with:

• title only
• abstract only
• title + abstract (recommended)

Limitations

• WoS Research Areas are journal-level labels, not article-level annotations.
  → This introduces noise, especially for multidisciplinary journals.
• Some Research Areas have low representation, affecting macro-F1.
• Multiclass classification forces each article into one label, even if it spans multiple disciplines.
• Not suitable for:
  • fine-grained WoS Categories (SCs)
  • ASJC Areas or MAG FoS models (use separate classifiers)
  • normative evaluation or ranking of research
  • decisions requiring verified human annotations

Predictions should be interpreted as high-level field approximations, not exact domain attribution.

Training and evaluation data

Training data consists of documents with Web of Science Research Area labels, derived from journal-level classification.
Because WoS SC/RAs are not publicly distributed, we rely on a curated dataset constructed from open-access publications where Research Areas can be inferred from the journal metadata.

Notes on WoS Research Areas

• Research Areas are high-level and non-hierarchical.
• Articles inherit the journal’s assigned Research Area(s).
• Some journals have multiple RAs, but this model uses a single predominant label for training.

Training procedure

Preprocessing

• Input text constructed as:
  title + ". " + abstract
• Tokenization using SPECTER2 tokenizer
• Maximum sequence length: 512 tokens

Model

• Base model: allenai/specter2_base
• Classification head: linear → softmax
• Loss: CrossEntropyLoss

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision Micro	Precision Macro	Recall Micro	Recall Macro	F1 Micro	F1 Macro
1.8283	1.0	8540	1.8278	0.5409	0.5409	0.5243	0.5409	0.5394	0.5409	0.5174
1.4971	2.0	17080	1.6945	0.5666	0.5666	0.5623	0.5666	0.5645	0.5666	0.5537
1.145	3.0	25620	1.7103	0.5774	0.5774	0.5676	0.5774	0.5747	0.5774	0.5636
0.8711	4.0	34160	1.7738	0.5770	0.5770	0.5753	0.5770	0.5739	0.5770	0.5681
0.6207	5.0	42700	1.8987	0.5763	0.5763	0.5744	0.5763	0.5741	0.5763	0.5691

Evaluation results

	precision	recall	f1-score	support
Acoustics	0.857143	0.75	0.8	40
Agriculture	0.446809	0.355932	0.396226	59
Allergy	0.823529	0.84	0.831683	50
Anatomy & Morphology	0.864865	0.744186	0.8	43
Anesthesiology	0.813953	0.625	0.707071	56
Anthropology	0.419355	0.26	0.320988	50
Archaeology	0.77551	0.655172	0.71028	58
Architecture	0.7	0.538462	0.608696	52
Area Studies	0.282051	0.244444	0.261905	45
Art	0.591549	0.75	0.661417	56
Arts & Humanities - Other Topics	0.333333	0.166667	0.222222	54
Asian Studies	0.361702	0.395349	0.377778	43
Astronomy & Astrophysics	0.836364	0.958333	0.893204	48
Automation & Control Systems	0.891304	0.836735	0.863158	49
Behavioral Sciences	0.578947	0.431373	0.494382	51
Biochemistry & Molecular Biology	0.428571	0.393443	0.410256	61
Biodiversity & Conservation	0.431034	0.595238	0.5	42
Biomedical Social Sciences	0.5625	0.661765	0.608108	68
Biophysics	0.708333	0.790698	0.747253	43
Biotechnology & Applied Microbiology	0.636364	0.368421	0.466667	57
Business & Economics	0.465753	0.73913	0.571429	46
Cardiovascular System & Cardiology	0.809524	0.708333	0.755556	48
Cell Biology	0.507463	0.586207	0.544	58
Chemistry	0.6	0.612245	0.606061	49
Classics	0.79661	0.854545	0.824561	55
Communication	0.454545	0.480769	0.46729	52
Computer Science	0.538462	0.528302	0.533333	53
Construction & Building Technology	0.631579	0.6	0.615385	40
Criminology & Penology	0.538462	0.792453	0.641221	53
Critical Care Medicine	0.66	0.611111	0.634615	54
Crystallography	1	0.979167	0.989474	48
Cultural Studies	0.25	0.142857	0.181818	49
Dance	0.581818	0.711111	0.64	45
Demography	0.530612	0.590909	0.55914	44
Dentistry, Oral Surgery & Medicine	0.574074	0.837838	0.681319	37
Dermatology	0.545455	0.679245	0.605042	53
Developmental Biology	0.590909	0.530612	0.55914	49
Education & Educational Research	0.363636	0.666667	0.470588	36
Electrochemistry	0.704918	0.826923	0.761062	52
Emergency Medicine	0.673077	0.744681	0.707071	47
Endocrinology & Metabolism	0.647059	0.634615	0.640777	52
Energy & Fuels	0.486842	0.787234	0.601626	47
Engineering	0.529412	0.62069	0.571429	58
Entomology	0.695652	0.711111	0.703297	45
Environmental Sciences & Ecology	0.342105	0.276596	0.305882	47
Ethnic Studies	0.615385	0.507937	0.556522	63
Evolutionary Biology	0.770833	0.74	0.755102	50
Family Studies	0.538462	0.651163	0.589474	43
Film, Radio & Television	0.571429	0.740741	0.645161	54
Fisheries	0.722222	0.604651	0.658228	43
Food Science & Technology	0.661765	0.681818	0.671642	66
Forestry	0.7	0.648148	0.673077	54
Gastroenterology & Hepatology	0.596491	0.755556	0.666667	45
General & Internal Medicine	0.473684	0.195652	0.276923	46
Genetics & Heredity	0.627907	0.586957	0.606742	46
Geochemistry & Geophysics	0.8125	0.52	0.634146	50
Geography	0.604167	0.557692	0.58	52
Geology	0.212121	0.145833	0.17284	48
Geriatrics & Gerontology	0.615385	0.603774	0.609524	53
Government & Law	0.529412	0.580645	0.553846	62
Health Care Sciences & Services	0.404255	0.413043	0.408602	46
Hematology	0.683333	0.82	0.745455	50
History	0.44186	0.431818	0.436782	44
History & Philosophy of Science	0.469388	0.425926	0.446602	54
Imaging Science & Photographic Technology	0.647059	0.647059	0.647059	51
Immunology	0.5	0.52381	0.511628	63
Infectious Diseases	0.5	0.547619	0.522727	42
Information Science & Library Science	0.519481	0.701754	0.597015	57
Instruments & Instrumentation	0.714286	0.581395	0.641026	43
Integrative & Complementary Medicine	0.724138	0.7	0.711864	60
International Relations	0.444444	0.489796	0.466019	49
Legal Medicine	0.854167	0.82	0.836735	50
Life Sciences & Biomedicine - Other Topics	0.666667	0.254545	0.368421	55
Linguistics	0.509434	0.627907	0.5625	43
Literature	0.431373	0.360656	0.392857	61
Marine & Freshwater Biology	0.5	0.553191	0.525253	47
Materials Science	0.291667	0.254545	0.271845	55
Mathematical & Computational Biology	0.547619	0.469388	0.505495	49
Mathematical Methods In Social Sciences	0.655738	0.634921	0.645161	63
Mathematics	0.584906	0.704545	0.639175	44
Mechanics	0.72093	0.553571	0.626263	56
Medical Ethics	0.542373	0.653061	0.592593	49
Medical Informatics	0.75	0.85	0.796875	60
Medical Laboratory Technology	0.7	0.5	0.583333	42
Metallurgy & Metallurgical Engineering	0.666667	0.680851	0.673684	47
Meteorology & Atmospheric Sciences	0.76	0.622951	0.684685	61
Microbiology	0.380282	0.6	0.465517	45
Microscopy	0.711538	0.637931	0.672727	58
Mineralogy	0.474359	0.672727	0.556391	55
Mining & Mineral Processing	0.609375	0.764706	0.678261	51
Music	0.733333	0.758621	0.745763	58
Mycology	0.807692	0.84	0.823529	50
Neurosciences & Neurology	0.5625	0.4	0.467532	45
Nuclear Science & Technology	0.672727	0.770833	0.718447	48
Nursing	0.688525	0.677419	0.682927	62
Nutrition & Dietetics	0.567568	0.446809	0.5	47
Obstetrics & Gynecology	0.373134	0.675676	0.480769	37
Oceanography	0.818182	0.734694	0.774194	49
Oncology	0.618182	0.548387	0.581197	62
Operations Research & Management Science	0.606061	0.689655	0.645161	58
Ophthalmology	0.803279	0.844828	0.823529	58
Optics	0.666667	0.625	0.645161	48
Orthopedics	0.462687	0.837838	0.596154	37
Otorhinolaryngology	0.656716	0.709677	0.682171	62
Paleontology	0.639344	0.684211	0.661017	57
Parasitology	0.482759	0.538462	0.509091	52
Pathology	0.384615	0.454545	0.416667	44
Pediatrics	0.5	0.638298	0.560748	47
Pharmacology & Pharmacy	0.487179	0.422222	0.452381	45
Philosophy	0.488889	0.511628	0.5	43
Physical Geography	0.651163	0.622222	0.636364	45
Physical Sciences - Other Topics	0.452381	0.44186	0.447059	43
Physics	0.714286	0.555556	0.625	54
Physiology	0.5625	0.5	0.529412	54
Plant Sciences	0.660377	0.686275	0.673077	51
Polymer Science	0.677966	0.784314	0.727273	51
Psychiatry	0.603774	0.744186	0.666667	43
Psychology	0.477273	0.381818	0.424242	55
Public Administration	0.476923	0.62	0.53913	50
Public, Environmental & Occupational Health	0	0	0	4
Radiology, Nuclear Medicine & Medical Imaging	0.645833	0.645833	0.645833	48
Rehabilitation	0.6	0.679245	0.637168	53
Religion	0.491525	0.604167	0.542056	48
Remote Sensing	0.5625	0.586957	0.574468	46
Reproductive Biology	0.736842	0.571429	0.643678	49
Research & Experimental Medicine	0.32	0.145455	0.2	55
Respiratory System	0.55814	0.648649	0.6	37
Rheumatology	0.729167	0.76087	0.744681	46
Robotics	0.555556	0.731707	0.631579	41
Science & Technology - Other Topics	0.142857	0.106383	0.121951	47
Social Issues	0.6	0.176471	0.272727	51
Social Sciences - Other Topics	0.153846	0.0677966	0.0941176	59
Social Work	0.392157	0.5	0.43956	40
Sociology	0.263158	0.0980392	0.142857	51
Spectroscopy	0.673469	0.717391	0.694737	46
Sport Sciences	0.714286	0.731707	0.722892	41
Substance Abuse	0.76	0.791667	0.77551	48
Surgery	0.363636	0.355556	0.359551	45
Technology - Other Topics	0.214286	0.0576923	0.0909091	52
Telecommunications	0.784314	0.727273	0.754717	55
Theater	0.648148	0.614035	0.630631	57
Thermodynamics	0.636364	0.446809	0.525	47
Toxicology	0.545455	0.782609	0.642857	46
Transplantation	0.853659	0.897436	0.875	39
Transportation	0.58209	0.75	0.655462	52
Tropical Medicine	0.4	0.425532	0.412371	47
Urban Studies	0.615385	0.489796	0.545455	49
Urology & Nephrology	0.622222	0.666667	0.643678	42
Veterinary Sciences	0.532258	0.647059	0.584071	51
Virology	0.710526	0.658537	0.683544	41
Water Resources	0.592593	0.615385	0.603774	52
Women's Studies	0.404255	0.413043	0.408602	46
Zoology	0.452381	0.487179	0.469136	39
accuracy	0.585616	0.585616	0.585616	0.585616
macro avg	0.578248	0.584159	0.572667	7592
weighted avg	0.583763	0.585616	0.576288	7592

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.1