Update README.md

README.md

@@ -8,7 +8,7 @@ tags:
 - rag
 - question-answering
 - fine-tuned
-- adalora
+- lora
 - liquid
 - extractive-qa
 datasets:
@@ -21,19 +21,19 @@ library_name: transformers
 pipeline_tag: question-answering
 ---
 
-# LFM2-1.2B-RAG Arabic (AdaLoRA Fine-tuned)
+# LFM2-1.2B-RAG Arabic (LoRA Fine-tuned)
 
-Fine-tuned version of [LiquidAI/LFM2-1.2B-RAG](https://huggingface.co/LiquidAI/LFM2-1.2B-RAG) for Arabic reading comprehension and question answering tasks using **AdaLoRA (Adaptive Low-Rank Adaptation)** technique.
+Fine-tuned version of [LiquidAI/LFM2-1.2B-RAG](https://huggingface.co/LiquidAI/LFM2-1.2B-RAG) for Arabic reading comprehension and question answering tasks using **LoRA (Low-Rank Adaptation)** technique.
 
 ## 📋 Model Description
 
-This model specializes in extractive question answering for Arabic text with adaptive parameter allocation. It has been fine-tuned on the Arabic Reading Comprehension Dataset (ARCD) using AdaLoRA, which dynamically adjusts the rank of different layers during training for optimal performance.
+This model specializes in extractive question answering for Arabic text. It has been fine-tuned using LoRA on the Arabic Reading Comprehension Dataset (ARCD) to improve its ability to answer questions based on provided context in Modern Standard Arabic.
 
 **Key Features:**
-- Optimized for Arabic extractive QA with adaptive rank allocation
-- Context-based question answering with high faithfulness
-- Balanced performance across multiple evaluation metrics
-- Parameter-efficient fine-tuning via AdaLoRA
+- Optimized for Arabic extractive QA
+- Context-based question answering
+- Maintains faithfulness to source documents
+- Efficient fine-tuning via LoRA (rank=16)
 
 ## 🎯 Intended Use
 
@@ -42,21 +42,19 @@ This model specializes in extractive question answering for Arabic text with ada
 - RAG (Retrieval-Augmented Generation) applications for Arabic content
 - Information extraction from Arabic documents
 - Educational tools for Arabic reading comprehension
-- Chatbots requiring grounded Arabic responses
 
 ### Downstream Use
 Can be further fine-tuned for:
 - Domain-specific QA (medical, legal, financial)
 - Multi-turn conversational QA
-- Cross-lingual QA systems
-- Document analysis pipelines
+- Document summarization with Q&A
 
 ### Out-of-Scope Use
 **Not recommended for:**
 - Open-domain question answering without context
-- Creative writing or story generation
-- Machine translation
-- Code generation or technical programming tasks
+- Creative writing or content generation
+- Translation tasks
+- Code generation
 
 ## 🚀 How to Use
 
@@ -67,7 +65,7 @@ import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
 # Load model and tokenizer
-model_id = "azeddinShr/LFM2-1.2B-RAG-ARABIC-AdaLoRA"
+model_id = "azeddinShr/LFM2-1.2B-RAG-ARABIC-LoRA"
 model = AutoModelForCausalLM.from_pretrained(
     model_id,
     device_map="auto",
@@ -100,8 +98,8 @@ with torch.no_grad():
 
 answer = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
 print(answer)  # Output: 500 مليار دولار
-```
 
+```
 
 ## 📊 Training Details
 
@@ -115,7 +113,7 @@ print(answer)  # Output: 500 مليار دولار
 
 ### Training Procedure
 
-**Fine-tuning method:** AdaLoRA (Adaptive Low-Rank Adaptation)
+**Fine-tuning method:** LoRA (Low-Rank Adaptation)
 
 **Hyperparameters:**
 - **Base model:** LiquidAI/LFM2-1.2B-RAG
@@ -126,18 +124,9 @@ print(answer)  # Output: 500 مليار دولار
 - **LR scheduler:** Cosine
 - **Warmup steps:** 50
 - **Weight decay:** 0.01
-
-**AdaLoRA Configuration:**
-- **Initial rank (r):** 16
-- **Target average rank:** 8
-- **Initial adapter rank:** 12
+- **LoRA rank (r):** 16
 - **LoRA alpha:** 32
 - **LoRA dropout:** 0.05
-- **Pruning start step (tinit):** 10% of total steps
-- **Pruning end step (tfinal):** 70% of total steps
-- **Pruning frequency (deltaT):** 10 steps
-- **Importance smoothing (beta1, beta2):** 0.85
-- **Orthogonality regularization:** 0.5
 - **Target modules:** w1, w2, w3, q_proj, k_proj, v_proj, out_proj, in_proj
 
 **Training infrastructure:**
@@ -145,38 +134,24 @@ print(answer)  # Output: 500 مليار دولار
 - Gradient checkpointing: Enabled
 - Framework: Hugging Face Transformers + PEFT + TRL
 
-## 🔒 Ethical Considerations
-
-- This model should not be used to generate misleading information or propaganda
-- Outputs should be fact-checked for critical applications
-- The model reflects statistical patterns in training data and may not represent complete or unbiased knowledge
-- Users are responsible for ensuring appropriate use in their applications
-
-## 🔬 Technical Details
 
-### What is AdaLoRA?
-
-AdaLoRA (Adaptive Low-Rank Adaptation) extends LoRA by dynamically allocating parameter budgets across different weight matrices based on their importance during training. This results in:
-- More efficient parameter usage
-- Better performance with fewer trainable parameters
-- Automatic pruning of less important adaptations
+## 🔒 Ethical Considerations
 
-### Advantages over standard LoRA
-- Adaptive rank allocation based on importance scores
-- Better performance-efficiency trade-off
-- More stable training dynamics
+- This model should not be used for generating misleading or false information
+- Users should verify factual claims, especially for sensitive topics
+- The model's responses reflect patterns in training data and may not represent complete or unbiased information
 
 ## 📜 Citation
 
 If you use this model in your research or application, please cite:
 
 ```bibtex
-@misc{lfm2-arabic-qa-adalora,
-  author = {Azeddin Sahir},
-  title = {LFM2-1.2B-RAG Arabic (AdaLoRA Fine-tuned)},
+@misc{lfm2-rag-arabic-lora,
+  author = {Azeddin sahir},
+  title = {LFM2-1.2B-RAG Arabic (LoRA Fine-tuned)},
   year = {2025},
   publisher = {Hugging Face},
-  howpublished = {\url{https://huggingface.co/azeddinShr/lfm2-1.2b-arabic-qa-adalora}}
+  howpublished = {\url{https://huggingface.co/azeddinShr/lfm2-1.2b-arabic-qa-lora}}
 }
 ```
 
@@ -185,16 +160,11 @@ If you use this model in your research or application, please cite:
 - **Base Model:** [LiquidAI](https://www.liquid.ai/) for LFM2-1.2B-RAG
 - **Dataset:** [ARCD](https://huggingface.co/datasets/hsseinmz/arcd) - Arabic Reading Comprehension Dataset
 - **Framework:** Hugging Face Transformers, PEFT, TRL
-- **Method:** AdaLoRA by Zhang et al.
 
 ## 📄 License
 
-Base model License
+Same as based model
 
 ## 📧 Contact
 
 For questions, issues, or collaboration opportunities, please open an issue in the model repository or contact via Hugging Face.
-
----
-
-**Note:** This is a research model. Always validate outputs for your specific use case and domain.