library_name: sentence-transformers
pipeline_tag: sentence-similarity
license: mit
base_model: BAAI/bge-large-en-v1.5
tags:
- sentence-transformers
- information-retrieval
- biomedical
- metasyn
datasets:
- THUIR/MetaSyn
MA-Retriever
MA-Retriever is the dense retriever released with MetaSyn. It fine-tunes
BAAI/bge-large-en-v1.5 for retrieving articles linked to the published
included-study lists of systematic reviews, scoping reviews, and
meta-analyses.
Resources
- Paper: arXiv:2606.17041
- Dataset: THUIR/MetaSyn
- Code and evaluator: THUIR/MetaSyn
Use
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BFTree/MA-Retriever")
def encode_protocol(research_question, population, intervention_or_exposure,
comparison, outcome):
fields = [
("Research Question", research_question),
("Population", population),
("Intervention or Exposure", intervention_or_exposure),
("Comparison", comparison),
("Outcome", outcome),
]
protocol = ". ".join(f"{name}: {value}" for name, value in fields if value)
query = "Represent this sentence for searching relevant passages: " + protocol
return model.encode(query, normalize_embeddings=True)
def encode_article(title, abstract):
document = f"Title: {title}. Abstract: {abstract}"
return model.encode(document, normalize_embeddings=True)
The protocol query concatenates the research question, Population, Intervention or Exposure, Comparison, and Outcome. It does not use the source-review title. Documents use title and abstract. Retrieval uses cosine similarity over normalized embeddings.
Training
The base revision is d4aa6901d3a41ba39fb536a557fa166f842b0e09. We used
40 reviews sampled from the training split to select the epoch count. Recall@20
is the primary metric, with Recall@100 and Recall@200 used as tie-breakers. The
released checkpoint was retrained for one epoch using the full training split
with Multiple Negatives Ranking Loss, batch size 64, learning rate 2e-5,
maximum sequence length 512, and seed 718. After excluding articles linked to
a test review, 334 of the 336 training reviews contribute positive pairs.
There are 5,585 constructed training pairs. The sentence-transformers training loader used 5,568 examples in complete batches during the epoch; this accounts for the smaller sample count shown by automatically generated trainer metadata.
Test retrieval
All metrics are macro-averaged over the 86 held-out reviews after removing the source review itself from each ranking.
| K | Recall@K | Precision@K |
|---|---|---|
| 5 | 24.0% | 47.9% |
| 10 | 38.8% | 42.0% |
| 20 | 53.5% | 34.0% |
| 50 | 75.3% | 22.6% |
| 100 | 84.2% | 14.0% |
| 200 | 91.7% | 8.4% |
The source-review split is disjoint, and every article linked to a test review is excluded as a positive retriever-training example.
Limitations
The model is trained on reviews from Nature Portfolio and a PubMed-centered corpus. Performance may not transfer unchanged to other databases, languages, or domains. A high-recall pool still requires protocol-based screening; model scores should not be treated as inclusion decisions.
Citation
@misc{metasyn2026,
title = {Benchmarking {LLM} Agents on Meta-Analysis Articles from {Nature} Portfolio},
author = {Anzhe Xie and Weihang Su and Yujia Zhou and Yiqun Liu and Min Zhang and Qingyao Ai},
year = {2026},
eprint = {2606.17041},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2606.17041}
}