--- base_model: Qwen/Qwen2.5-0.5B-Instruct library_name: peft pipeline_tag: text-generation license: apache-2.0 language: - en tags: - base_model:adapter:Qwen/Qwen2.5-0.5B-Instruct - peft - lora - transformers - safetensors - text-generation - conversational - nigeria - nigerian-pidgin - business-assistant model-index: - name: GaiaLab Naija Adapter v0.2 results: - task: type: text-generation name: Text Generation metrics: - type: eval_loss value: 1.3829631805419922 name: Best Validation Loss --- # GaiaLab Naija Adapter v0.2 GaiaLab Naija Adapter v0.2 is an experimental LoRA adapter fine-tuned from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct). The project explores how a compact open-weight language model can be adapted for clearer, more locally relevant communication for Nigerian users, including Nigerian Pidgin, practical business questions, education, technology, and everyday assistance. This repository contains a **PEFT LoRA adapter**, not a standalone model. It must be loaded together with the original Qwen base model. ## Model Details ### Model Description GaiaLab Naija Adapter v0.2 was trained as part of the GaiaLab Naija Assistant project. The adapter was developed to investigate culturally and linguistically relevant AI assistance for Nigerian communities while preserving important English business and technical terminology. Version 0.2 represents an early research and engineering milestone. It demonstrates a complete workflow covering dataset preparation, validation, deterministic train-validation splitting, LoRA fine-tuning, checkpoint selection, adapter export, and inference. The model currently produces useful English responses, but its Nigerian Pidgin behavior is not yet consistently strong. A larger and more linguistically diverse training dataset is planned for future releases. - **Developed by:** Oluwafemi Idiakhoa - **Organization:** GaiaLab AI - **Model type:** PEFT LoRA causal language model adapter - **Base architecture:** Qwen2.5 0.5B Instruct - **Language:** English, with experimental Nigerian Pidgin adaptation - **License:** Apache License 2.0 - **Fine-tuned from:** `Qwen/Qwen2.5-0.5B-Instruct` - **Adapter version:** v0.2 - **Adapter format:** Safetensors - **Human evaluation status:** Pending ### Model Sources - **Adapter:** `mgbam/gaialab-naija-adapter-v0.2` - **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) - **Source repository:** [GaiaLab Naija Assistant](https://github.com/oluwafemidiakhoa/gaialab-naija-assistant) - **Development branch:** `v0.2-development` ## Uses ### Direct Use The adapter may be used for experimental text generation involving: - general Nigerian-focused assistance; - small-business explanations; - educational support; - technology guidance; - conversational question answering; - English-to-Nigerian-Pidgin adaptation experiments; - evaluation of low-resource language fine-tuning; - research on culturally relevant AI systems. ### Downstream Use The adapter may be integrated into: - research demonstrations; - conversational applications; - educational prototypes; - Nigerian small-business assistants; - multilingual and code-switching experiments; - local inference applications; - larger retrieval-augmented generation systems. Developers should perform additional evaluation before using the adapter in a public or production-facing application. ### Out-of-Scope Use This model should not be relied upon for: - medical diagnosis or treatment; - legal advice; - financial or investment decisions; - emergency response; - identity verification; - high-risk government decisions; - fully automated employment or lending decisions; - unsupervised production deployment; - harmful, fraudulent, deceptive, or illegal activity. The model should not be presented as an authoritative source of Nigerian law, policy, culture, language, or professional guidance. ## Bias, Risks, and Limitations This is an early experimental adapter trained on only 200 records. Known limitations include: - inconsistent use of natural Nigerian Pidgin; - a tendency to respond in standard English even when Pidgin is requested; - limited coverage of Nigerian regions, dialects, professions, and social groups; - possible hallucination or unsupported claims; - limited handling of long or highly technical prompts; - inherited biases and limitations from the Qwen base model; - possible overrepresentation of business-oriented examples; - weak generalization beyond the training domains; - limited evidence from human evaluation; - no formal safety benchmark has yet been completed. Nigerian Pidgin varies by region, community, age group, and context. The adapter should not be treated as representing one universally correct form of Nigerian Pidgin. ### Recommendations Users and downstream developers should: - keep a human reviewer involved in high-impact use cases; - verify factual claims independently; - avoid using the model as a professional authority; - test performance across Nigerian regions and user groups; - evaluate both English and Nigerian Pidgin responses; - monitor hallucination, harmful output, and cultural misrepresentation; - clearly disclose that responses are AI-generated; - use deterministic evaluation prompts when comparing model versions. ## How to Get Started Install the required packages: ```bash pip install -U transformers peft accelerate torch Load the adapter with its base model: import torch from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer BASE_MODEL_ID = "Qwen/Qwen2.5-0.5B-Instruct" ADAPTER_ID = "mgbam/gaialab-naija-adapter-v0.2" tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID) base_model = AutoModelForCausalLM.from_pretrained( BASE_MODEL_ID, dtype=torch.float16, device_map="auto", ) model = PeftModel.from_pretrained( base_model, ADAPTER_ID, ) model.eval() messages = [ { "role": "system", "content": ( "You are GaiaLab Naija Assistant. Respond clearly and naturally. " "Use Nigerian Pidgin when requested while preserving important " "business and technical terms." ), }, { "role": "user", "content": ( "Explain in Nigerian Pidgin why small businesses should keep " "proper financial records." ), }, ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, ) inputs = tokenizer( prompt, return_tensors="pt", ).to(model.device) with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=180, temperature=0.7, top_p=0.9, do_sample=True, repetition_penalty=1.1, ) generated_tokens = outputs[0][inputs["input_ids"].shape[1]:] response = tokenizer.decode( generated_tokens, skip_special_tokens=True, ) print(response) For more reproducible evaluation, disable sampling: outputs = model.generate( **inputs, max_new_tokens=180, do_sample=False, ) Training Details Training Data The adapter was trained on the GaiaLab Naija v0.2 dataset. Dataset statistics: Item Value Total validated records 200 Training records 180 Validation records 20 Validation ratio 10% Duplicate records after validation 0 Dataset format JSONL Human evaluation completed No Training dataset path used during development: data/v0.2/prepared/gaialab_naija_v0.2_combined.jsonl The dataset contains instruction-response examples intended to support Nigerian-focused assistance, practical communication, business guidance, and experimental Nigerian Pidgin generation. The dataset is still small and should not be considered comprehensive or fully representative of Nigerian language use. Training Procedure The training workflow included: JSONL schema validation; duplicate detection; semantic record validation; deterministic seeded shuffling; 90/10 train-validation split; chat-template formatting; LoRA fine-tuning; evaluation after each epoch; best-checkpoint selection using validation loss; export of the best PEFT adapter. Preprocessing Training and validation records were: loaded from a validated JSONL dataset; formatted using the Qwen chat template; tokenized with the base-model tokenizer; filtered to remove unusable examples; split deterministically using seed 42. Training Hyperparameters Hyperparameter Value Base model Qwen/Qwen2.5-0.5B-Instruct Epochs 3 Learning rate 0.0002 Per-device batch size 2 Gradient accumulation steps 8 Effective batch size 16 LoRA rank 16 LoRA alpha 32 Random seed 42 Validation ratio 0.10 Evaluation frequency Every epoch Optimized modules q_proj, k_proj, v_proj, o_proj Adapter format Safetensors Speeds, Sizes, and Times Metric Value Training runtime 111.0862 seconds Training samples per second 4.861 Training steps per second 0.324 Global training steps 36 Final epoch 3.0 Reported total FLOPs 143,137,130,188,800 Adapter weight size Approximately 8.68 MB Best checkpoint checkpoint-36 Evaluation Testing Data, Factors, and Metrics Validation Data The validation set contained 20 records selected through a deterministic 90/10 split from the validated 200-record dataset. The validation data was used during training for checkpoint selection. It should not be treated as a comprehensive external benchmark. Evaluation Factors Future human evaluation should examine: Nigerian Pidgin naturalness; factual correctness; instruction following; cultural relevance; clarity; usefulness; business terminology preservation; code-switching quality; hallucination frequency; safety and harmful-output behavior. Metrics The automated training metric currently reported is cross-entropy validation loss. Validation loss measures prediction performance on the held-out validation records. A lower value indicates improved fit to the validation set, but it does not independently prove that the model produces natural Nigerian Pidgin or more useful answers. Results Epoch Validation Loss 1 1.577 2 1.423 3 1.382963 Additional final training metrics: Metric Value Final training loss 1.733298 Best validation loss 1.382963 Best checkpoint checkpoint-36 Global step 36 Summary Validation loss improved across all three epochs, indicating that the adapter learned from the training data. However, early qualitative testing showed that the adapter may still respond in standard English when explicitly asked to use Nigerian Pidgin. Therefore, the numerical evaluation should not be interpreted as proof of strong Nigerian Pidgin capability. A structured human evaluation comparing the base model, v0.1 adapter, and v0.2 adapter is still required. Model Examination Initial qualitative evaluation suggests that v0.2: generates concise and generally understandable responses; can provide practical business explanations; avoids obvious hallucination in some basic prompts; does not yet consistently follow Nigerian Pidgin instructions; requires additional linguistically rich training data. These observations are preliminary and are not a substitute for a formal benchmark. Environmental Impact A formal carbon-emissions estimate was not recorded for this experiment. Hardware type: CUDA-enabled cloud GPU Cloud provider: Google Colab Training duration: Approximately 111 seconds Compute region: Not recorded GPU model: Not recorded in the training summary Carbon emitted: Not measured Because the adapter was trained on a 0.5B-parameter base model for only 36 optimization steps, the training run was relatively small. However, no verified emissions figure is currently available. Technical Specifications Model Architecture and Objective The adapter modifies selected attention projection layers in Qwen2.5-0.5B-Instruct using Low-Rank Adaptation. Target modules: q_proj k_proj v_proj o_proj The training objective was supervised causal language modeling over instruction-response conversations. The adapter does not contain the complete base-model weights. The original Qwen base model must be loaded separately. Compute Infrastructure Hardware Training required a CUDA-enabled GPU. The exact GPU model was not preserved in the training summary. Software Key software components included: Python PyTorch Transformers PEFT TRL Datasets Accelerate Safetensors Framework Versions PEFT: 0.19.1 Base model: Qwen2.5-0.5B-Instruct Other package versions may vary depending on the inference environment. Reproducibility The training configuration used: model: Qwen/Qwen2.5-0.5B-Instruct dataset: data/v0.2/prepared/gaialab_naija_v0.2_combined.jsonl learning_rate: 0.0002 epochs: 3 batch_size: 2 gradient_accumulation: 8 lora_rank: 16 lora_alpha: 32 target_modules: - q_proj - k_proj - v_proj - o_proj evaluation_frequency: 1 seed: 42 Training command: python train_adapter.py \ --config training/v0.2_config.yaml \ --output-dir outputs/gaialab-adapter-v0.2 \ --validation-ratio 0.10 Citation No formal paper has yet been published for this model. Suggested citation: BibTeX @software{idiakhoa2026gaialabnaija, author = {Oluwafemi Idiakhoa}, title = {GaiaLab Naija Adapter v0.2}, year = {2026}, organization = {GaiaLab AI}, publisher = {Hugging Face}, url = {https://huggingface.co/mgbam/gaialab-naija-adapter-v0.2} } APA Idiakhoa, O. (2026). GaiaLab Naija Adapter v0.2 [LoRA language model adapter]. GaiaLab AI. Hugging Face. Glossary LoRA: Low-Rank Adaptation, a parameter-efficient method for fine-tuning language models. PEFT: Parameter-Efficient Fine-Tuning. Adapter: A small set of learned weights applied to a larger base model. Nigerian Pidgin: A widely used English-based contact language spoken across Nigeria. Validation loss: A numerical measure of model prediction error on held-out data. Code-switching: Moving between languages or language varieties within a conversation. Future Work Planned improvements include: expanding the dataset beyond 200 examples; adding more natural Nigerian Pidgin conversations; improving regional and demographic coverage; building a dedicated evaluation benchmark; comparing the base model, v0.1, and v0.2 adapters; performing structured human evaluation; evaluating safety and hallucination behavior; documenting dataset provenance and annotation procedures; training a future v0.3 adapter with substantially more examples. Model Card Authors Oluwafemi Idiakhoa Founder and CEO, GaiaLab AI Model Card Contact For project information, issues, or contributions, use the GaiaLab Naija Assistant GitHub repository: https://github.com/oluwafemidiakhoa/gaialab-naija-assistant This version presents v0.2 honestly as a successful experimental adapter while clearly documenting that its Nigerian Pidgin performance still requires improvement and formal human evaluation.