license: cc-by-sa-4.0
language:
- ar
base_model:
- U4RASD/NeoAraBERT
tags:
- neoarabert
- neobert
- bert
- MSA
- Dialect
- masked-language-model
- custom_code
pipeline_tag: feature-extraction
library_name: Transformers
NeoAraBERT
NeoAraBERT is a state-of-the-art open-source Arabic text-embedding model built on the NeoBERT architecture. We pretrain NeoAraBERT on diverse open-source and internal datasets covering modern standard, classical, and dialectal Arabic. We guided our design choices with Arabic tailored ablation studies including text normalization, light stemming, and diacritics-aware tokenization handling. We also performed POS-aware token masking and learning-rate scheduling ablation studies. We benchmarked NeoAraBERT against five top-performing Arabic models on 23 tasks, including a synonym-based task, Muradif, that directly assesses embedding quality with no additional fine-tuning. NeoAraBERT variants rank first in 18 tasks and improve average performance across the full benchmark suite.
This is the NeoAraBERT_Mix checkpoint, our best-performing checkpoint overall. This model was introduced at the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026). For more information, visit our website: https://www.acrps.ai/neoarabert.
How to Use
Install these libraries:
pip install fast-disambig torch==2.5.1 transformers xformers==0.0.28.post3
Load the model and use it to generate embeddings:
from transformers import AutoModel, AutoTokenizer
model_name = "U4RASD/NeoAraBERT"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)
# Tokenize input text
text = "المركز العربيّ للأبحاث ودراسة السياسات."
inputs = tokenizer(text, return_tensors="pt")
# Generate embeddings
outputs = model(**inputs)
embedding = outputs.last_hidden_state[:, 0, :]
print(embedding.shape)
Citation
If you use the code, model, or the Muradif benchmark, please reference this work in your paper:
The citation will be added here soon.
License
This model is licensed under the CC BY-SA 4.0 license. The text of the license can be found here.