README.md

---
base_model: Qwen/Qwen2.5-0.5B-Instruct
library_name: peft
pipeline_tag: text-generation
license: apache-2.0
language:
  - en
tags:
  - base_model:adapter:Qwen/Qwen2.5-0.5B-Instruct
  - peft
  - lora
  - transformers
  - safetensors
  - text-generation
  - conversational
  - nigeria
  - nigerian-pidgin
  - business-assistant
model-index:
  - name: GaiaLab Naija Adapter v0.2
    results:
      - task:
          type: text-generation
          name: Text Generation
        metrics:
          - type: eval_loss
            value: 1.3829631805419922
            name: Best Validation Loss
---

# GaiaLab Naija Adapter v0.2

GaiaLab Naija Adapter v0.2 is an experimental LoRA adapter fine-tuned from
[Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).

The project explores how a compact open-weight language model can be adapted for clearer, more locally relevant communication for Nigerian users, including Nigerian Pidgin, practical business questions, education, technology, and everyday assistance.

This repository contains a **PEFT LoRA adapter**, not a standalone model. It must be loaded together with the original Qwen base model.

## Model Details

### Model Description

GaiaLab Naija Adapter v0.2 was trained as part of the GaiaLab Naija Assistant project. The adapter was developed to investigate culturally and linguistically relevant AI assistance for Nigerian communities while preserving important English business and technical terminology.

Version 0.2 represents an early research and engineering milestone. It demonstrates a complete workflow covering dataset preparation, validation, deterministic train-validation splitting, LoRA fine-tuning, checkpoint selection, adapter export, and inference.

The model currently produces useful English responses, but its Nigerian Pidgin behavior is not yet consistently strong. A larger and more linguistically diverse training dataset is planned for future releases.

- **Developed by:** Oluwafemi Idiakhoa
- **Organization:** GaiaLab AI
- **Model type:** PEFT LoRA causal language model adapter
- **Base architecture:** Qwen2.5 0.5B Instruct
- **Language:** English, with experimental Nigerian Pidgin adaptation
- **License:** Apache License 2.0
- **Fine-tuned from:** `Qwen/Qwen2.5-0.5B-Instruct`
- **Adapter version:** v0.2
- **Adapter format:** Safetensors
- **Human evaluation status:** Pending

### Model Sources

- **Adapter:** `mgbam/gaialab-naija-adapter-v0.2`
- **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
- **Source repository:** [GaiaLab Naija Assistant](https://github.com/oluwafemidiakhoa/gaialab-naija-assistant)
- **Development branch:** `v0.2-development`

## Uses

### Direct Use

The adapter may be used for experimental text generation involving:

- general Nigerian-focused assistance;
- small-business explanations;
- educational support;
- technology guidance;
- conversational question answering;
- English-to-Nigerian-Pidgin adaptation experiments;
- evaluation of low-resource language fine-tuning;
- research on culturally relevant AI systems.

### Downstream Use

The adapter may be integrated into:

- research demonstrations;
- conversational applications;
- educational prototypes;
- Nigerian small-business assistants;
- multilingual and code-switching experiments;
- local inference applications;
- larger retrieval-augmented generation systems.

Developers should perform additional evaluation before using the adapter in a public or production-facing application.

### Out-of-Scope Use

This model should not be relied upon for:

- medical diagnosis or treatment;
- legal advice;
- financial or investment decisions;
- emergency response;
- identity verification;
- high-risk government decisions;
- fully automated employment or lending decisions;
- unsupervised production deployment;
- harmful, fraudulent, deceptive, or illegal activity.

The model should not be presented as an authoritative source of Nigerian law, policy, culture, language, or professional guidance.

## Bias, Risks, and Limitations

This is an early experimental adapter trained on only 200 records.

Known limitations include:

- inconsistent use of natural Nigerian Pidgin;
- a tendency to respond in standard English even when Pidgin is requested;
- limited coverage of Nigerian regions, dialects, professions, and social groups;
- possible hallucination or unsupported claims;
- limited handling of long or highly technical prompts;
- inherited biases and limitations from the Qwen base model;
- possible overrepresentation of business-oriented examples;
- weak generalization beyond the training domains;
- limited evidence from human evaluation;
- no formal safety benchmark has yet been completed.

Nigerian Pidgin varies by region, community, age group, and context. The adapter should not be treated as representing one universally correct form of Nigerian Pidgin.

### Recommendations

Users and downstream developers should:

- keep a human reviewer involved in high-impact use cases;
- verify factual claims independently;
- avoid using the model as a professional authority;
- test performance across Nigerian regions and user groups;
- evaluate both English and Nigerian Pidgin responses;
- monitor hallucination, harmful output, and cultural misrepresentation;
- clearly disclose that responses are AI-generated;
- use deterministic evaluation prompts when comparing model versions.

## How to Get Started

Install the required packages:

```bash
pip install -U transformers peft accelerate torch

Load the adapter with its base model:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL_ID = "Qwen/Qwen2.5-0.5B-Instruct"
ADAPTER_ID = "mgbam/gaialab-naija-adapter-v0.2"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)

base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_ID,
)

model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are GaiaLab Naija Assistant. Respond clearly and naturally. "
            "Use Nigerian Pidgin when requested while preserving important "
            "business and technical terms."
        ),
    },
    {
        "role": "user",
        "content": (
            "Explain in Nigerian Pidgin why small businesses should keep "
            "proper financial records."
        ),
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(
    prompt,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=180,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
        repetition_penalty=1.1,
    )

generated_tokens = outputs[0][inputs["input_ids"].shape[1]:]

response = tokenizer.decode(
    generated_tokens,
    skip_special_tokens=True,
)

print(response)

For more reproducible evaluation, disable sampling:

outputs = model.generate(
    **inputs,
    max_new_tokens=180,
    do_sample=False,
)
Training Details
Training Data

The adapter was trained on the GaiaLab Naija v0.2 dataset.

Dataset statistics:

Item	Value
Total validated records	200
Training records	180
Validation records	20
Validation ratio	10%
Duplicate records after validation	0
Dataset format	JSONL
Human evaluation completed	No

Training dataset path used during development:

data/v0.2/prepared/gaialab_naija_v0.2_combined.jsonl

The dataset contains instruction-response examples intended to support Nigerian-focused assistance, practical communication, business guidance, and experimental Nigerian Pidgin generation.

The dataset is still small and should not be considered comprehensive or fully representative of Nigerian language use.

Training Procedure

The training workflow included:

JSONL schema validation;
duplicate detection;
semantic record validation;
deterministic seeded shuffling;
90/10 train-validation split;
chat-template formatting;
LoRA fine-tuning;
evaluation after each epoch;
best-checkpoint selection using validation loss;
export of the best PEFT adapter.
Preprocessing

Training and validation records were:

loaded from a validated JSONL dataset;
formatted using the Qwen chat template;
tokenized with the base-model tokenizer;
filtered to remove unusable examples;
split deterministically using seed 42.
Training Hyperparameters
Hyperparameter	Value
Base model	Qwen/Qwen2.5-0.5B-Instruct
Epochs	3
Learning rate	0.0002
Per-device batch size	2
Gradient accumulation steps	8
Effective batch size	16
LoRA rank	16
LoRA alpha	32
Random seed	42
Validation ratio	0.10
Evaluation frequency	Every epoch
Optimized modules	q_proj, k_proj, v_proj, o_proj
Adapter format	Safetensors
Speeds, Sizes, and Times
Metric	Value
Training runtime	111.0862 seconds
Training samples per second	4.861
Training steps per second	0.324
Global training steps	36
Final epoch	3.0
Reported total FLOPs	143,137,130,188,800
Adapter weight size	Approximately 8.68 MB
Best checkpoint	checkpoint-36
Evaluation
Testing Data, Factors, and Metrics
Validation Data

The validation set contained 20 records selected through a deterministic 90/10 split from the validated 200-record dataset.

The validation data was used during training for checkpoint selection. It should not be treated as a comprehensive external benchmark.

Evaluation Factors

Future human evaluation should examine:

Nigerian Pidgin naturalness;
factual correctness;
instruction following;
cultural relevance;
clarity;
usefulness;
business terminology preservation;
code-switching quality;
hallucination frequency;
safety and harmful-output behavior.
Metrics

The automated training metric currently reported is cross-entropy validation loss.

Validation loss measures prediction performance on the held-out validation records. A lower value indicates improved fit to the validation set, but it does not independently prove that the model produces natural Nigerian Pidgin or more useful answers.

Results
Epoch	Validation Loss
1	1.577
2	1.423
3	1.382963

Additional final training metrics:

Metric	Value
Final training loss	1.733298
Best validation loss	1.382963
Best checkpoint	checkpoint-36
Global step	36
Summary

Validation loss improved across all three epochs, indicating that the adapter learned from the training data.

However, early qualitative testing showed that the adapter may still respond in standard English when explicitly asked to use Nigerian Pidgin. Therefore, the numerical evaluation should not be interpreted as proof of strong Nigerian Pidgin capability.

A structured human evaluation comparing the base model, v0.1 adapter, and v0.2 adapter is still required.

Model Examination

Initial qualitative evaluation suggests that v0.2:

generates concise and generally understandable responses;
can provide practical business explanations;
avoids obvious hallucination in some basic prompts;
does not yet consistently follow Nigerian Pidgin instructions;
requires additional linguistically rich training data.

These observations are preliminary and are not a substitute for a formal benchmark.

Environmental Impact

A formal carbon-emissions estimate was not recorded for this experiment.

Hardware type: CUDA-enabled cloud GPU
Cloud provider: Google Colab
Training duration: Approximately 111 seconds
Compute region: Not recorded
GPU model: Not recorded in the training summary
Carbon emitted: Not measured

Because the adapter was trained on a 0.5B-parameter base model for only 36 optimization steps, the training run was relatively small. However, no verified emissions figure is currently available.

Technical Specifications
Model Architecture and Objective

The adapter modifies selected attention projection layers in Qwen2.5-0.5B-Instruct using Low-Rank Adaptation.

Target modules:

q_proj
k_proj
v_proj
o_proj

The training objective was supervised causal language modeling over instruction-response conversations.

The adapter does not contain the complete base-model weights. The original Qwen base model must be loaded separately.

Compute Infrastructure
Hardware

Training required a CUDA-enabled GPU. The exact GPU model was not preserved in the training summary.

Software

Key software components included:

Python
PyTorch
Transformers
PEFT
TRL
Datasets
Accelerate
Safetensors
Framework Versions
PEFT: 0.19.1
Base model: Qwen2.5-0.5B-Instruct

Other package versions may vary depending on the inference environment.

Reproducibility

The training configuration used:

model: Qwen/Qwen2.5-0.5B-Instruct
dataset: data/v0.2/prepared/gaialab_naija_v0.2_combined.jsonl
learning_rate: 0.0002
epochs: 3
batch_size: 2
gradient_accumulation: 8
lora_rank: 16
lora_alpha: 32
target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj
evaluation_frequency: 1
seed: 42

Training command:

python train_adapter.py \
  --config training/v0.2_config.yaml \
  --output-dir outputs/gaialab-adapter-v0.2 \
  --validation-ratio 0.10
Citation

No formal paper has yet been published for this model.

Suggested citation:

BibTeX
@software{idiakhoa2026gaialabnaija,
  author       = {Oluwafemi Idiakhoa},
  title        = {GaiaLab Naija Adapter v0.2},
  year         = {2026},
  organization = {GaiaLab AI},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/mgbam/gaialab-naija-adapter-v0.2}
}
APA

Idiakhoa, O. (2026). GaiaLab Naija Adapter v0.2 [LoRA language model adapter]. GaiaLab AI. Hugging Face.

Glossary
LoRA: Low-Rank Adaptation, a parameter-efficient method for fine-tuning language models.
PEFT: Parameter-Efficient Fine-Tuning.
Adapter: A small set of learned weights applied to a larger base model.
Nigerian Pidgin: A widely used English-based contact language spoken across Nigeria.
Validation loss: A numerical measure of model prediction error on held-out data.
Code-switching: Moving between languages or language varieties within a conversation.
Future Work

Planned improvements include:

expanding the dataset beyond 200 examples;
adding more natural Nigerian Pidgin conversations;
improving regional and demographic coverage;
building a dedicated evaluation benchmark;
comparing the base model, v0.1, and v0.2 adapters;
performing structured human evaluation;
evaluating safety and hallucination behavior;
documenting dataset provenance and annotation procedures;
training a future v0.3 adapter with substantially more examples.
Model Card Authors

Oluwafemi Idiakhoa
Founder and CEO, GaiaLab AI

Model Card Contact

For project information, issues, or contributions, use the GaiaLab Naija Assistant GitHub repository:

https://github.com/oluwafemidiakhoa/gaialab-naija-assistant


This version presents v0.2 honestly as a successful experimental adapter while clearly documenting that its Nigerian Pidgin performance still requires improvement and formal human evaluation.