README.md

---
base_model: unsloth/gemma-3n-E4B-unsloth-bnb-4bit
library_name: peft
language:
- bm
tags:
- gemma3n
- lora
- peft
- unsloth
- bambara
---

# gemma-3n-bm-base

A LoRA adapter for Gemma 3n E4B — the pretrained, non-instruction-tuned variant — trained for
Bambara with [Unsloth](https://github.com/unslothai/unsloth) and TRL.

Adapter weights only. Load onto `unsloth/gemma-3n-E4B-unsloth-bnb-4bit`; that checkpoint is
4-bit NF4, and the adapter should be loaded onto the same quantised base it was trained against.

## Config

| | |
| --- | --- |
| Rank `r` | 128 |
| `lora_alpha` | 32 |
| `use_rslora` | `true` — effective scale is `alpha / sqrt(r)` ≈ 2.83, not `alpha / r` |
| `lora_dropout` | 0 |

Adapted: the attention and MLP projections of the 35 decoder layers, the audio tower's attention
projections, and — unusually for a LoRA — `embed_tokens` and `lm_head`, which is what makes this
the vocabulary-adaptation stage.

## Usage

```python
import torch
from transformers import AutoProcessor, Gemma3nForConditionalGeneration
from peft import PeftModel

base = Gemma3nForConditionalGeneration.from_pretrained(
    "unsloth/gemma-3n-E4B-unsloth-bnb-4bit",
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)
model = PeftModel.from_pretrained(base, "djelia/gemma-3n-bm-base")
model.eval()

processor = AutoProcessor.from_pretrained("djelia/gemma-3n-bm-base", padding_side="left")

inputs = processor(text="Bamako ye ", return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(processor.decode(out[0], skip_special_tokens=True))
```

## Notes

This adapter sits on the pretrained base, so use plain text continuation rather than chat
formatting.

`bitsandbytes` and `accelerate` are required for the 4-bit base. On `transformers` releases
older than the `dtype=` rename, pass `torch_dtype=torch.bfloat16`.