---
base_model: unsloth/qwen2.5-7b-unsloth-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/qwen2.5-7b-unsloth-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
license: apache-2.0
language:
- ms
---
```markdown
---
base_model: unsloth/qwen2.5-7b-unsloth-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/qwen2.5-7b-unsloth-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
- malaysian
- bahasa-melayu
- atlasflux
---

# Model Card for AtlasFlux Qwen 2.5 7B – LoRA Adapter for Malaysian Context

<!-- Provide a quick summary of what the model is/does. -->

This is a **LoRA adapter** fine-tuned on `unsloth/qwen2.5-7b-unsloth-bnb-4bit` to improve understanding of Malaysian Bahasa Melayu, colloquial slang (Manglish), and regional dialects (Kelantan, Kedah, Terengganu, Johor, Sabah, Sarawak). The adapter is designed for chatbots and AI applications serving Malaysian users.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

The model was fine-tuned using **QLoRA (4-bit quantisation)** on a custom instruction‑response dataset of **2,968 examples** covering general knowledge, local culture, everyday conversations, and the personal branding of AtlasFlux AI and Rainspeed Labs. The base model is `unsloth/qwen2.5-7b-unsloth-bnb-4bit`, an optimised 4‑bit version of Qwen2.5‑7B. Training was done on a single Google Colab T4 GPU (16GB VRAM) in about 90 minutes.

- **Developed by:** Muhammad Nabil (Rainspeed Labs / AtlasFlux AI)
- **Funded by:** Self‑funded
- **Shared by:** Muhammad Nabil
- **Model type:** LoRA adapter for decoder‑only transformer
- **Language(s) (NLP):** Bahasa Melayu (Standard, Colloquial, Regional dialects) and English
- **License:** Apache 2.0
- **Finetuned from model:** `unsloth/qwen2.5-7b-unsloth-bnb-4bit`

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** [rainspeed/atlasflux-qwen-7b-1.0](https://huggingface.co/rainspeed/atlasflux-qwen-7b-1.0)
- **Paper:** Included as `README.md` and research paper within the repo
- **Demo:** Not yet hosted – see usage instructions

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Load the adapter with the base model for text generation. **Required prompt format:**

```
### Instruction:\n{user question}\n\n### Response:\n
```

Example code:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen2.5-7b-unsloth-bnb-4bit",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("unsloth/qwen2.5-7b-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base, "rainspeed/atlasflux-qwen-7b-1.0")

prompt = "### Instruction:\nSiapa yang membina AtlasFlux AI?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

### Downstream Use

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

The adapter can be merged into the base model for full‑weight deployment, or used as‑is with PEFT. Suitable for Malaysian customer support bots, localised Q&A, and educational tools.

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

- Generating harmful, discriminatory, or illegal content
- High‑stakes decisions (medical, legal, financial) without human verification
- Unauthorised commercial use that violates Apache 2.0

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

The model was fine‑tuned on a **small dataset (2,968 examples)**, which may cause limited coverage of certain dialects or topics, occasional repetition, factual errors (hallucinations), and biases present in the training data (e.g., from public forums). No explicit safety alignment was performed.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should always validate important outputs before acting on them. For production systems, consider augmenting with retrieval‑augmented generation (RAG) to ground answers in trusted sources. Periodically evaluate model outputs on representative Malaysian user inputs.

## How to Get Started with the Model

Use the code below to get started with the model.

See the `Direct Use` section above for inference. To merge the adapter into a full model:

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("unsloth/qwen2.5-7b-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base, "rainspeed/atlasflux-qwen-7b-1.0")
merged = model.merge_and_unload()
merged.save_pretrained("atlasflux_merged")
```

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

- **Size:** 2,968 instruction‑response pairs
- **Sources:** Public online forums, social media (Twitter, Facebook public pages, Lowyat.net), synthetic generation using Qwen2.5‑3B
- **Composition:** ~70% Standard Bahasa Melayu, ~20% Colloquial slang (Manglish), ~10% Regional dialects (Kelantan, Kedah, Terengganu, Johor, Sabah, Sarawak)
- **Format:** JSONL with fields `instruction` and `response`
- **Preprocessing:** Manual cleaning to remove personally identifiable information (PII) and noise

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing

Prompts were formatted as `### Instruction:\n{instruction}\n\n### Response:\n{response}`. The dataset was split into training only; no validation split was used.

#### Training Hyperparameters

- **Training regime:** fp16 mixed precision
- **LoRA rank (r):** 16
- **LoRA alpha:** 16
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- **Dropout:** 0.0
- **Bias:** none
- **Gradient checkpointing:** enabled (Unsloth)
- **Per‑device batch size:** 1
- **Gradient accumulation steps:** 8
- **Learning rate:** 2e‑4
- **Optimiser:** AdamW 8‑bit
- **Warmup steps:** 5
- **Max steps:** 500
- **Random seed:** 3407

#### Speeds, Sizes, Times

- **Trainable parameters:** ~20.2 million (0.26% of full model)
- **Training time:** ~90 minutes on Google Colab T4 GPU
- **Adapter size:** ~80 MB (safetensors)
- **Full merged model size:** ~15 GB (FP16)

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

A held‑out set of 100 Malaysian prompts (not seen during training) was used for qualitative evaluation.

#### Factors

Performance was assessed on factual recall of personal branding (AtlasFlux, Rainspeed Labs, Muhammad Nabil), colloquial slang, and dialect responses.

#### Metrics

Qualitative assessment only (no quantitative benchmarks due to resource constraints).

### Results

The model correctly answered:
- “Siapa yang membina AtlasFlux AI?” → returns factual answer with name, UiTM, course
- “Aku nak gi mana?” (Kelantan dialect) → appropriate dialect response
- “Line internet aku slow gila.” → helpful troubleshooting advice

Limitations: small dataset leads to occasional generic English answers for very rare dialect phrases.

#### Summary

The fine‑tuned adapter successfully improves Malaysian cultural and linguistic understanding over the base Qwen2.5‑7B model, within the scope of a small‑scale fine‑tuning project.

## Model Examination

<!-- Relevant interpretability work for the model goes here -->

Not performed.

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** NVIDIA T4
- **Hours used:** 1.5 hours
- **Cloud Provider:** Google Colab (assumed US‑central region)
- **Compute Region:** US‑central (estimated)
- **Carbon Emitted:** ~0.06 kg CO₂ (approx.)

## Technical Specifications

### Model Architecture and Objective

Base architecture: Qwen 2.5‑7B (28 layers, 7.6B parameters). LoRA matrices injected into Q, K, V, O, and MLP projection layers. Training objective: next‑token prediction (causal language modelling).

### Compute Infrastructure

#### Hardware

- GPU: NVIDIA T4 (16 GB VRAM)
- RAM: 12 GB (Colab runtime)
- Disk: ~78 GB temporary storage

#### Software

- Python 3.12
- PyTorch 2.1.0
- Transformers 4.36.0
- PEFT 0.12.0
- Unsloth 2026.5.9
- bitsandbytes 0.49.2
- accelerate 0.25.0

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

```bibtex
@misc{atlasflux2026,
  author       = {Muhammad Nabil},
  title        = {AtlasFlux Qwen 2.5 7B – LoRA adapter for Malaysian cultural and linguistic contexts},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/rainspeed/atlasflux-qwen-7b-1.0}
}
```

**APA:**

Muhammad Nabil. (2026). *AtlasFlux Qwen 2.5 7B – LoRA adapter for Malaysian contexts*. Hugging Face. https://huggingface.co/rainspeed/atlasflux-qwen-7b-1.0

## Glossary

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information

[More Information Needed]

## Model Card Authors

Muhammad Nabil (Rainspeed Labs / AtlasFlux AI)

## Model Card Contact

support.atlasflux@gmail.com or via ticket system at [ai.atlasflux.my](https://ai.atlasflux.my)

### Framework versions

- PEFT 0.19.1
```