NOC-Lab/AbCDR-ESM2

Antibody ESM2 Paired Model

Model Description

This model is a fine-tuned version of ESM2-3B for paired antibody sequences (heavy and light chains).

Key Features:

• Trained on paired antibody sequences
• 15% whole-chain (WC) masking followed by 50% CDR masking during fine-tuning
• Input format: Heavy-Light chains separated by "-"
• Output: 2560-dimensional embeddings
• Optimized for antibody CDR region understanding

Preprocessing

Sequences were:

1. Combined as: HEAVY-LIGHT (with "-" separator)
2. Tokenized with ESM2 tokenizer
3. CDR regions annotated for masking

Usage

Loading the Model

from transformers import EsmModel, AutoTokenizer
import torch

# Load model and tokenizer
model = EsmModel.from_pretrained("NOC-Lab/AbCDR-ESM2")
tokenizer = AutoTokenizer.from_pretrained("NOC-Lab/AbCDR-ESM2")
model.eval()

Extract Embeddings

# Prepare paired sequence
SEP_TOKEN = "-" 
heavy_chain = (
    "EVQLVESGGGLVQPGGSLRLSCAASGFTFSSYAMSWVRQAPGKGLEWVAVISYDGSNKYYADSVKGRF"
    "TISADTSKNTAYLQMNSLRAEDTAVYYCAREGYYGSSYWYFDYWGQGTLVTVSS"
)
light_chain = (
    "DIQMTQSPSSLSASVGDRVTITCRASQSISSYLNWYQQKPGKAPKLLIYAASSLQSGVPSRFSGSGS"
    "GTDFTLTISSLQPEDFATYYCQQSYSTPLTFGGGTKVEIK"
)
paired_sequence = f"{heavy_chain}{SEP_TOKEN}{light_chain}"

# Tokenize
inputs = tokenizer(paired_sequence, return_tensors="pt", add_special_tokens=True)

# Extract embeddings
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state
    
# Mean pooling
mask = inputs["attention_mask"].unsqueeze(-1)
pooled = (embeddings * mask).sum(1) / mask.sum(1)

print(f"Embedding shape: {pooled.shape}")  # (1, 2560)

Input Format

Required Format: HEAVY_CHAIN-LIGHT_CHAIN

• Heavy and light chains must be separated by hyphen (-)
• Use standard single-letter amino acid codes
• No spaces in sequence
• Uncommon residues should be replaced with X

Example:

sequence = "EVQLVESGGGLVQPGGSLRLSCAASGFTFSSYAMS...-DIQMTQSPSSLSASVGDRVTITCRASQSISS..."

Output

• Embedding dimension: 2560
• Sequence length: Variable (up to ~1024 tokens including special tokens)
• Format: PyTorch tensor

Publication

Preferential CDR masking in paired antibody language models improves binding affinity prediction

Mahtab Talaei, Kenji C. Walker, Boran Hao, Eliot Jolley, Yeping Jin, Dima Kozakov, John Misasi, Sandor Vajda, Ioannis Ch. Paschalidis, Diane Joseph-McCarthy.

Communications AI & Computing 1, 7 (2026).

DOI: https://doi.org/10.1038/s44488-026-00010-2

Published: August 13, 2026

Read the published article

Citation

If you use this model, please cite the published paper:

@article{Talaei2026,
  author = {Talaei, Mahtab and Walker, Kenji C. and Hao, Boran and Jolley, Eliot and Jin, Yeping and Kozakov, Dima and Misasi, John and Vajda, Sandor and Paschalidis, Ioannis Ch. and Joseph-McCarthy, Diane},
  title = {Preferential {CDR} masking in paired antibody language models improves binding affinity prediction},
  journal = {Communications AI \& Computing},
  volume = {1},
  pages = {7},
  year = {2026},
  doi = {10.1038/s44488-026-00010-2}
}

Contact

• Maintainer: Network Optimization & Control (NOC) Lab
• Email: mtalaei@bu.edu
• GitHub: https://github.com/noc-lab/AbCDR-ESM
• Paper: Communications AI & Computing

License

This model is released under the MIT License.

Acknowledgments

• Base model: ESM2 by Meta AI
• Data: OAS database

---

Note: For private repositories, you'll need to authenticate:

# Option 1: CLI login
huggingface-cli login

# Option 2: Environment variable
export HF_TOKEN="your_token_here"