---
license: apache-2.0
language:
- ar
- en
pipeline_tag: text-generation
tags:
  - pytorch
library_name: transformers
---
# Fanar-1-9B-Instruct

**Fanar-1-9B-Instruct** is a powerful Arabic-English LLM developed by [Qatar Computing Research Institute (QCRI)](https://www.hbku.edu.qa/en/qcri) and [Hamad Bin Khalifa University (HBKU)](https://www.hbku.edu.qa/). It is the instruction-tuned version of [Fanar-1-9B](). Fanar continually pretrains the `google/gemma-2-9b` model on 1T Arabic and English tokens. Fanar pays particular attention to the richness of the Arabic language by supporting a diverse set of Arabic dialects including Modern Standard Arabic (MSA), Levantine, and Egyptian. Fanar, through meticulous curation of the pretraining and instruction-tuning data, is aligned with Arab cultural values.

We have published a comprehensive [report](https://arxiv.org/pdf/2501.13944) with all the details regarding FANAR. We also provide an API to the model (request access [here](https://api.fanar.qa/request/en)).

---

## Model Details

| Attribute                  | Value                              |
|---------------------------|------------------------------------|
| Developed by              | [QCRI](https://www.hbku.edu.qa/en/qcri) and [HBKU](https://www.hbku.edu.qa/)                      |
| Model Type                | Autoregressive Transformer         |
| Parameter Count           | 8.7 Billion                          |
| Context Length            | 4096 Tokens                        |
| Input                     | text only                          |
| Output                    | text only                          |
| Training Framework        | [LitGPT](https://github.com/Lightning-AI/litgpt) |
| Pretraining Token Count   | 1 Trillion (ar + en) |
| SFT Instructions          | 4.5M                               |
| DPO Preference Pairs      | 250K                               |
| Languages                 | Arabic, English                    |
| License                   | Apache 2.0                         |
<!-- | Precision                 | bfloat16                           | -->

---

## Model Training

#### Pretraining
Fanar was continually pretrained on 1T tokens, with a balanced focus on Arabic and English: ~515B English tokens from a carefully curated subset of the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset, 410B Arabic tokens that we collected, parsed, and flitered from a variety of sources, 102B code tokens curated from [The Stack](https://github.com/bigcode-project/the-stack-v2) dataset. Our codebase used the [LitGPT](https://github.com/Lightning-AI/litgpt) framework. 

#### Post-training
Fanar underwent a two-phase post-training pipeline:

| Phase | Size |
|-------|------|
| Supervised Fine-tuning (SFT) | 4.5M Instructions |
| Direct Preference Optimization (DPO) | 250K Preference Pairs |

---


## Getting Started

Fanar is compatible with the Hugging Face `transformers` library (≥ v4.40.0). Here's how to load and use the model:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "QCRI/Fanar-1-9B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [
    {"role": "user", "content": "ما هي عاصمة قطر؟"},
]

inputs = tokenizer.apply_chat_template(messages, tokenize=False, return_tensors="pt")
outputs = model.generate(**tokenizer(inputs, return_tensors="pt", return_token_type_ids=False), max_new_tokens=256)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

## Intended Use

Fanar-1-9B-Instruct is built for:

- Conversational agents (Arabic only or bilingual)
- Cultural and dialectal question answering in Arabic
- Educational, governmental, and civic NLP applications in the Arab world
- Research on Arabic natural language generation and understanding

Fanar is intended to be deployed as part of a broader AI system. Developers are encouraged to implement proper safeguards to ensure culturally respectful, accurate, and safe deployment. It should not be used to generate or spread **harmful, illegal, or misleading content**

`A version of this model is currently deployed as part of a real-world system at (chat.fanar.qa).`

--- 

## Ethical Considerations & Limitations

Fanar is capable of generating fluent and contextually appropriate responses, but as with any generative model there are uncertainities. The model may produce **biased, offensive, or incorrect outputs**. The model should It is **not suitable for high-stakes decision making** (e.g., legal, medical, or financial advice). Though we have extensively tested Fanar and attempted to mitigate these issues, we cannot redress every possible scenario. Thus, we advise developers to implement safety checks and perform domain-specific fine-tuning for sensitive use cases.

The output generated by this model is not considered a statement of QCRI, HBKU, or any other organization or individual.
 
---

## Evaluation

Evaluation results for Fanar-1-9B-Instruct will be released soon across Arabic and English benchmarks including:

- Arabic MMLU
- Ar-IFEval
- TruthfulQA
- HellaSwag
- ARC (Easy/Challenge)
- OpenBookQA

Evaluation was conducted using a modified version of the LM Evaluation Harness and internal cultural alignment benchmarks.

---

## Citation

If you use Fanar in your research or applications, please cite:

```bibtex
@misc{fanarllm2025,
      title={Fanar: An Arabic-Centric Multimodal Generative AI Platform}, 
      author={Fanar Team},
      year={2025},
      url={https://arxiv.org/abs/2501.13944}, 
}
```

---

## Acknowledgements

This project is a collaboration between [Qatar Computing Research Institute (QCRI)](https://qcri.org) and [Hamad Bin Khalifa University (HBKU)](https://hbku.edu.qa). We thank our engineering and research teams for their efforts in advancing Arabic-centric large language models.

---

## License

This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).