TwiGen-Qwen — Asante Twi Instruction-Following LoRA
LoRA adapter fine-tuned on Qwen3.5-0.8B to follow instructions and respond in Asante Twi (Akan).
Description
This is a PEFT LoRA adapter (rank=32, alpha=64) trained on ~495k Twi instruction-response pairs. It takes an instruction (and optional English source text) and generates a response entirely in Twi.
Built as part of the TwiGen project. A MiniCPM4-8B version is also available at twi-gen-minicpm4.
Live demo
- TwiGen Space: https://huggingface.co/spaces/ghananlpcommunity/twi-gen
- TTS for generated output available via nano-twi
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = "Qwen/Qwen3.5-0.8B"
adapter = "ghananlpcommunity/twi-gen-qwen"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
base,
torch_dtype=torch.bfloat16,
device_map="cuda:0",
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()
instruction = "Write a short story about a farmer and his harvest"
messages = [
{"role": "user", "content": instruction + "\n\nRespond entirely in Twi. Your response should not exceed 250 words."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=2048).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
Training data
- Dataset: ghananlpcommunity/twi-instructions (private)
- 495,000 instruction-response pairs covering translation, summarisation, storytelling, question answering, and general content generation — all in Twi.
- 2 epochs with cosine LR decay (peak 2e-4), LoRA rank 32, alpha 64.
- Training loss: 0.5205, final step loss: 0.4313.
Model details
| Property | Value |
|---|---|
| Base model | Qwen3.5-0.8B |
| Adapter size | ~43 MB |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training precision | bfloat16 |
| VRAM (inference) | ~1.8 GB (with base model) |
Limitations
- Fine-tuned for Twi responses only. Instructions in English are preferred but Twi instructions also work.
- Maximum context length: 2048 tokens.
- May occasionally produce English words or phrases — the "Respond entirely in Twi" system instruction helps.
- Not safety-aligned; use with appropriate guardrails for production.
License
Apache 2.0
- Downloads last month
- -