docs: add model/dataset card

README.md

@@ -5,11 +5,483 @@ language:
 base_model:
 - FacebookAI/xlm-roberta-base
 pipeline_tag: fill-mask
+tags:
+- xlm-roberta
+- fill-mask
+- masked-lm
+- bambara
+- bamanankan
+- mali
+- vocabulary-extension
 ---
 
-# Model Card for Model ID
+# bm-xlm-roberta-base
 
-This model is a fine-tuned version of [FacebookAI/xlm-roberta-base](https://huggingface.co/FacebookAI/xlm-roberta-base) on an 11M words Bambara dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.1160
-- Accuracy: 0.7599
+A Bambara (Bamanankan, `bm`) adaptation of **XLM-RoBERTa base**, continued-pretrained with the
+masked-language-modelling objective and shipped with a **vocabulary-extended tokenizer** (253,421
+tokens vs. 250,002 in stock XLM-R).
+
+> [!IMPORTANT]
+> **This is a masked-LM checkpoint, not a task model.** The architecture is
+> `XLMRobertaForMaskedLM`. It predicts `<mask>` and produces contextual representations. It does
+> **not** classify, tag, translate, or embed sentences out of the box. Use it as an *initialisation*
+> for fine-tuning on sequence classification, token classification (NER/POS), extractive QA, or as
+> the encoder backbone of a retrieval model.
+
+The headline reason to prefer this over stock `FacebookAI/xlm-roberta-base` for Bambara is
+tokenization: on a 5,000-sentence Bambara sample it produces **30.0% fewer tokens** (measurement and
+method below), which directly translates to shorter sequences, more real context inside the 512-token
+window, and cheaper fine-tuning.
+
+## Model details
+
+| | |
+|---|---|
+| Repository | `djelia/bm-xlm-roberta-base` (public) |
+| Architecture | `XLMRobertaForMaskedLM` (encoder-only, absolute position embeddings) |
+| Parameters | 280,924,397 (all `F32`) |
+| Layers / hidden / heads / FFN | 12 / 768 / 12 / 3072 |
+| Activation | `gelu` |
+| `max_position_embeddings` | 514 (usable sequence length **512**; `model_max_length: 512`) |
+| `vocab_size` | **253,421** |
+| `type_vocab_size` | 1 |
+| `tie_word_embeddings` | `true` (LM head decoder is tied to the input embedding matrix) |
+| Special token ids | `<s>`=0, `<pad>`=1, `</s>`=2, `<unk>`=3, **`<mask>`=253420** |
+| Files | `config.json`, `model.safetensors` (1,123,721,924 B), `tokenizer.json` (17,001,775 B), `tokenizer_config.json` |
+| Total repo storage | 1,140,723,699 B (~1.14 GB) |
+| Created / last modified | 2026-02-04 (both; all four commits authored by `oza75`) |
+| `transformers_version` in config | `5.0.0` |
+
+All of the above is read from the repo's `config.json`, `tokenizer_config.json`, and the Hub API
+model record.
+
+### Lineage
+
+The repo is tagged `base_model: FacebookAI/xlm-roberta-base`, which is the ultimate ancestor: the
+first 250,001 vocabulary entries and their ids are inherited unchanged from stock XLM-R (verified by
+an ordered vocabulary comparison, see below).
+
+The immediate provenance is a training run in the author's personal namespace. The
+`model.safetensors` in this repo is **byte-identical** (SHA-256
+`025cd2563f9aa71c26999eed79c0115b400ce29bbe033a9026312032a2bbd9c2`) to the one in
+[`oza75/xlm-roberta-bambara-01`](https://huggingface.co/oza75/xlm-roberta-bambara-01). I compared the
+LFS SHA-256 of `model.safetensors` across every `oza75/*xlm-roberta*bambara*` repo; that one matched
+and the other six did not. This repo was uploaded first (2026-02-04); the `oza75` copy was published
+later (2026-04-23) *with* the Trainer-generated metadata that this repo omits.
+
+The sibling repo declares its own base as `oza75/xlm-roberta-bambara-base-01`, which in turn declares
+`oza75/xlm-roberta-bambara-initial` — so the real chain is a multi-stage adaptation
+(`FacebookAI/xlm-roberta-base` → vocabulary extension + continued pretraining stages → this
+checkpoint), not a single fine-tune. So this is **not** a re-publication of a different model: it is
+the same artifact as `oza75/xlm-roberta-bambara-01`, published to the org namespace first.
+
+## Tokenizer: what the 3,419 added tokens actually are
+
+This is the most substantive difference from stock XLM-R, so it is worth stating precisely. I
+downloaded both `tokenizer.json` files and compared them directly.
+
+**Structure of the change.** The stock 250,002-entry vocabulary is preserved *in its original order*
+at ids 0–250,000. The 3,419 new tokens were **appended** at ids 250,001–253,419, and stock's `<mask>`
+was **moved from id 250,001 to id 253,420** (the only ordered mismatch in the overlapping range). No
+stock token was removed.
+
+> [!WARNING]
+> **Token ids are not interchangeable with stock XLM-R.** Ids 0–250,000 match, but `<mask>` moved
+> from 250001 → 253420. Never mix this tokenizer with a stock XLM-R checkpoint or vice versa. Always
+> load the tokenizer from this repo.
+
+**Composition of the 3,419 added tokens** (my count over `tokenizer.json`):
+
+| Property | Count | Share |
+|---|---|---|
+| Word-initial (begin with the `▁` metaspace marker) | 2,541 | 74.3% |
+| Contain a Bambara-specific letter (`ɛ ɔ ɲ ŋ` / `Ɛ Ɔ Ɲ Ŋ`) | 1,333 | 39.0% |
+| Pure ASCII-Latin (e.g. `▁taa`, `▁jama`, `▁kelen`, `baara`, `muso`) | 2,042 | 59.7% |
+| Punctuation-bearing (`▁,`, `▁.`, `▁?..`, `▁....`, and the odd `##`) | 38 | 1.1% |
+| Digit-bearing | 0 | 0% |
+| Single-character (`ԑ`, `Ɛ`, `Ŋ`, `Ɲ`, and a stray `̸`) | 5 | 0.1% |
+
+Token length (stripped of `▁`) peaks at 4–5 characters; the additions are overwhelmingly Bambara
+*words and morphemes*, not characters. Representative examples: `▁bɛ`, `▁tɛ`, `▁kɛ`, `▁bɛɛ`, `▁fɔ`,
+`▁mɔgɔ`, `▁jama`, `▁kelen`, `▁yɔrɔ`, `▁cogo`, `▁baara`, `▁muso`, plus productive suffix pieces
+(`kɔrɔ`, `tigi`, `ya`, `nba`) and Malian proper nouns (`▁Ɲafunke`, `▁Ɔsitarali`, `▁Ɛsipaɲi`).
+
+**So it is both.** The additions are dominated by Bambara whole words (which stock XLM-R has no
+entries for at all), and they *also* close a genuine character-coverage hole:
+
+> The capital letters **`Ɛ` (U+0190), `Ɲ` (U+01F8), and `Ŋ` (U+014A) are absent from the stock
+> XLM-R vocabulary entirely** — I confirmed that stock encodes each of them to `<unk>` (id 3).
+> Lowercase `ɛ ɔ ɲ ŋ` and capital `Ɔ` are present in stock, but only as isolated single characters,
+> so stock shreds every Bambara word containing them.
+
+A concrete side-by-side (`add_special_tokens=True`):
+
+```
+Input: "Mɔgɔw bɛɛ bɛ bange hɔrɔnya ni danbe ni josira lakanani na."
+
+this repo (20 tokens):
+  <s> ▁Mɔgɔ w ▁bɛɛ ▁bɛ ▁bange ▁hɔrɔn ya ▁ni ▁dan be ▁ni ▁jo sira ▁laka na ni ▁na . </s>
+
+stock XLM-R (30 tokens):
+  <s> ▁M ɔ g ɔ w ▁b ɛ ɛ ▁b ɛ ▁bang e ▁h ɔ r ɔ nya ▁ni ▁dan be ▁ni ▁jos ira ▁la kan ani ▁na . </s>
+```
+
+### Measured tokenization benefit
+
+**This is my own measurement, not a figure taken from any published card.** Method, stated so it is
+reproducible:
+
+- **Corpus:** [`djelia/bambara-texts`](https://huggingface.co/datasets/djelia/bambara-texts),
+  file `data/train-00000-of-00001.parquet`, column `text` (84,328 rows).
+- **Sample:** rows with non-null, non-empty `text`, then
+  `pandas.Series.sample(n=5000, random_state=0)` — a fixed seed, so the exact sample is reproducible.
+- **Tokenizers:** `tokenizers.Tokenizer.from_file()` on `tokenizer.json` from this repo and from
+  `FacebookAI/xlm-roberta-base`, `encode_batch(..., add_special_tokens=False)`.
+- **Sample size:** 4,614,792 characters / 917,625 whitespace-delimited words.
+
+| Metric | `djelia/bm-xlm-roberta-base` | stock `xlm-roberta-base` |
+|---|---|---|
+| Total tokens | **1,454,892** | 2,079,639 |
+| Tokens per word | **1.585** | 2.266 |
+| Characters per token | **3.172** | 2.219 |
+| `<unk>` tokens (id 3) | **48** (0.0033%) | 1,382 (0.0665%) |
+| Sentences containing ≥1 `<unk>` | **9** / 5,000 | 556 / 5,000 |
+
+**Corpus-level reduction: 30.04% fewer tokens.** Per sentence, the reduction is a median of 30.00%
+(mean 28.78%, p10 20.72%, p90 35.43%), and **this tokenizer was shorter-or-equal on 100.0% of the
+5,000 sentences** — there is no regression tail.
+
+A useful sanity check on whether the new vocabulary is actually load-bearing: **29.38% of all tokens
+produced on this Bambara corpus come from the 3,419 added ids.** 1.3% of the vocabulary carries
+nearly 30% of the token mass.
+
+> [!NOTE]
+> The corpus is a mixed-provenance Bambara collection (largest sources: `saillab/alpaca-bambara-cleaned`,
+> `bigscience/xP3all`, `cis-lmu/GlotCC-V1`, `facebook/belebele`). The 30% figure describes *this*
+> distribution of Bambara text. Orthographically non-standard or heavily code-switched Bambara will
+> land differently.
+
+### Tokenizer caveats
+
+> [!WARNING]
+> **The Unicode normalizer was replaced, and it no longer normalizes Unicode.** Stock XLM-R uses
+> SentencePiece's `Precompiled` charsmap (NFKC-style folding). This tokenizer instead uses
+> `Sequence[Strip(right), Replace(" {2,}" → "▁")]`. Two observed consequences:
+>
+> - **NFC vs. NFD input tokenizes differently.** `"ka nà"` in NFC → `▁n`, `à`; the same string in
+>   NFD → `▁na`, `̀`. Stock produces the identical result for both. **Normalize your text to NFC
+>   yourself** before tokenizing, at both fine-tuning and inference time, or you will get silent
+>   train/serve skew.
+> - **Compatibility characters are not folded.** `①` and `fi` pass through as-is (stock maps them to
+>   `1` and `fi`).
+
+> [!WARNING]
+> `tokenizer_config.json` declares `"tokenizer_class": "XLMRobertaTokenizer"` and `"use_fast": false`,
+> but the repo ships **only `tokenizer.json`** — there is no `sentencepiece.bpe.model`. I verified the
+> tokenizer loads correctly on **`transformers` 5.14.1**. On older `transformers` 4.x, where
+> `use_fast: false` would attempt to build the SentencePiece-backed slow tokenizer, loading may fail;
+> pass `use_fast=True` explicitly if you hit that.
+
+## Are the new embeddings actually trained?
+
+Yes. This matters — a vocabulary-extended checkpoint whose new rows were never trained is worthless —
+so I checked it rather than assuming.
+
+The embedding matrix **was resized correctly**: `roberta.embeddings.word_embeddings.weight` has shape
+`[253421, 768]`, read from the `model.safetensors` header. Four independent pieces of evidence say the
+added rows carry learned signal:
+
+1. **Magnitude is inconsistent with fresh initialisation.** Rows 250,001–253,420 have element-wise
+   σ = 0.1157 and mean L2 norm 3.204 (σ 0.273). `config.json` sets `initializer_range: 0.02`, so a
+   freshly-initialised row would have norm ≈ 0.02·√768 ≈ **0.55** — off by ~6×.
+2. **Norms track token frequency.** Correlation between a token's unigram score in `tokenizer.json`
+   and its embedding norm is **r = 0.818** over the added ids (vs. r = 0.277 over the stock ids).
+   Randomly initialised rows cannot correlate with corpus frequency.
+3. **The LM head bias for the new ids is non-zero and varied**: over ids 250,001+, mean −0.011,
+   σ 0.122, range [−2.644, 0.347], with **zero** exactly-zero entries.
+4. **Nearest neighbours are semantically coherent** (cosine similarity over the full L2-normalised
+   embedding matrix):
+
+   | Token | Top neighbours |
+   |---|---|
+   | `▁bɛ` ("is") | `▁tɛ` (0.75) — *its own negative form* — `▁Bɛ`, `bɛ`, `▁Nbɛ` |
+   | `▁mɔgɔ` ("person") | `▁Mogo`, `mɔgɔ`, `▁muso` ("woman"), `▁Krɛcɛn` |
+   | `▁yɔrɔ` ("place") | `▁Cɔgɔ`/`▁cogo` ("manner"), `▁waati` ("time"), `▁Yɔrɔ` |
+   | `▁kelen` ("one") | `kelen`, `▁Saaba` ("three"), `▁wɔɔrɔ` ("six"), `▁Kelen` |
+   | `▁Ɲafunke` (Malian town) | a tight cluster of other Malian toponyms at 0.79–0.83 — `▁Direyi`, `▁Yuwaru`, `▁Lere`, `▁Duwanza`, `▁Kemasina`, `▁Burɛmu` |
+
+   Numerals cluster with numerals, toponyms with toponyms, and a copula with its own negation. That
+   structure cannot arise from noise.
+
+> [!NOTE]
+> One observation worth recording without over-reading it: the added rows sit at a systematically
+> **lower norm than the stock rows** (mean 3.204 vs. 5.505) and are much tighter (σ 0.273 vs. 0.432).
+> This is the expected signature of embeddings that received fewer gradient updates than the
+> pretrained ones — they are trained, but for less wall-clock than the original XLM-R vocabulary. No
+> action is implied; it is simply the shape of the checkpoint.
+
+## Usage
+
+### Fill-mask
+
+XLM-R's mask token is **`<mask>`** (not `[MASK]`). In this checkpoint its id is **253420**.
+
+```python
+from transformers import pipeline
+
+fill = pipeline("fill-mask", model="djelia/bm-xlm-roberta-base")
+
+# "All people are born in freedom __ dignity."  (expected fill: "ni" = "and")
+for p in fill("Mɔgɔw bɛɛ bɛ bange hɔrɔnya <mask> danbe la."):
+    print(f"{p['score']:.4f}  {p['token_str']!r}  ->  {p['sequence']}")
+```
+
+Or explicitly, without the pipeline:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+
+model_id = "djelia/bm-xlm-roberta-base"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForMaskedLM.from_pretrained(model_id).eval()
+
+text = "Mɔgɔw bɛɛ bɛ bange hɔrɔnya <mask> danbe la."
+inputs = tokenizer(text, return_tensors="pt")
+
+with torch.no_grad():
+    logits = model(**inputs).logits
+
+mask_index = (inputs.input_ids == tokenizer.mask_token_id).nonzero()[0, 1]
+top = logits[0, mask_index].topk(5)
+
+for score, token_id in zip(top.values.tolist(), top.indices.tolist()):
+    print(f"{score:7.3f}  {tokenizer.decode([token_id])!r}")
+```
+
+Both snippets were executed against this checkpoint (`transformers` 5.14.1, `torch` 2.13.0). The
+actual output for that sentence — the opening of the UDHR Article 1 in Bambara, *"every person is
+born in freedom \_\_ dignity"* — is:
+
+| Rank | Token | Probability |
+|---|---|---|
+| 1 | `ni` ("and") | 0.875 |
+| 2 | `ani` ("and", alt. form) | 0.097 |
+| 3 | `,` | 0.004 |
+| 4 | `kɔnɔ` ("in/inside") | 0.003 |
+| 5 | `la` (locative postposition) | 0.003 |
+
+The top two predictions are both correct Bambara conjunctions, and rank 1 is the word actually used
+in the source text — a sanity check that the checkpoint is intact and the mask token is right, not a
+benchmark.
+
+### Contextual embeddings
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModel
+
+tokenizer = AutoTokenizer.from_pretrained("djelia/bm-xlm-roberta-base")
+encoder = AutoModel.from_pretrained("djelia/bm-xlm-roberta-base").eval()
+
+batch = tokenizer(["Aw ni ce.", "I ka kɛnɛ wa?"], padding=True, return_tensors="pt")
+with torch.no_grad():
+    hidden = encoder(**batch).last_hidden_state  # (batch, seq_len, 768)
+```
+
+> [!WARNING]
+> **Do not use `pooler_output`.** This checkpoint contains **no pooler weights**. I diffed its tensor
+> key set against stock `FacebookAI/xlm-roberta-base` (after normalising the `gamma`/`beta` naming
+> below) and the *only* difference is that `roberta.pooler.dense.weight` and
+> `roberta.pooler.dense.bias` are **absent here**. Confirmed at runtime: `AutoModel.from_pretrained`
+> reports `missing_keys = {'pooler.dense.weight', 'pooler.dense.bias'}`, so those two tensors are
+> **randomly initialised** and `pooler_output` is meaningless noise. Use `last_hidden_state` and pool
+> it yourself. (`AutoModel` also reports the five `lm_head.*` tensors as unexpected — that is normal
+> and harmless when loading an MLM checkpoint into a bare encoder.)
+
+> [!WARNING]
+> `last_hidden_state` gives *token* representations. Mean-pooling them is **not** a sentence-embedding
+> model — no contrastive objective was ever applied here. For retrieval, fine-tune with a
+> sentence-similarity objective (e.g. `sentence-transformers`) on top of this encoder.
+
+### Fine-tuning starting point
+
+Swap in whichever head your task needs; the encoder weights load, the head is initialised fresh.
+
+```python
+from transformers import (
+    AutoModelForSequenceClassification,  # or AutoModelForTokenClassification
+    AutoTokenizer,
+    DataCollatorWithPadding,
+    Trainer,
+    TrainingArguments,
+)
+
+MODEL_ID = "djelia/bm-xlm-roberta-base"
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
+model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, num_labels=3)
+
+def preprocess(batch):
+    # Normalize to NFC first — this tokenizer does no Unicode normalization of its own.
+    import unicodedata
+    texts = [unicodedata.normalize("NFC", t) for t in batch["text"]]
+    return tokenizer(texts, truncation=True, max_length=512)
+
+tokenized = your_dataset.map(preprocess, batched=True)
+
+trainer = Trainer(
+    model=model,
+    args=TrainingArguments(
+        output_dir="bm-xlmr-finetuned",
+        learning_rate=2e-5,          # standard XLM-R base range: 1e-5 .. 3e-5
+        per_device_train_batch_size=16,
+        num_train_epochs=3,
+        warmup_ratio=0.06,
+        weight_decay=0.01,
+        fp16=True,
+    ),
+    train_dataset=tokenized["train"],
+    eval_dataset=tokenized["validation"],
+    data_collator=DataCollatorWithPadding(tokenizer),
+)
+trainer.train()
+```
+
+Two practical notes:
+
+- **Do not resize the embeddings** unless you are adding your own special tokens. The matrix is
+  already 253,421 rows and matches the shipped tokenizer.
+- Because the added embeddings are less-trained than the inherited ones (see above), leaving the
+  embedding matrix **unfrozen** during fine-tuning is the safer default.
+
+## Training
+
+> [!NOTE]
+> This repo ships **no** training artifacts (no `trainer_state.json`, `training_args.bin`, or
+> `all_results.json`). Everything in this section is read from
+> [`oza75/xlm-roberta-bambara-01`](https://huggingface.co/oza75/xlm-roberta-bambara-01), whose
+> `model.safetensors` is byte-identical to this one (SHA-256 verified, see *Lineage*). It is therefore
+> the same run — but it is second-hand for *this* repo, and is labelled as such.
+
+The original card in this repo states the model was trained "on an 11M words Bambara dataset"
+(author's claim, preserved verbatim; the dataset itself is not identified).
+
+**Reported evaluation results** (MLM objective, held-out split of the same unnamed corpus):
+
+| Metric | Value |
+|---|---|
+| Eval loss | 1.1160 |
+| Eval masked-token accuracy | 0.7599 |
+| Eval perplexity | 3.0525 |
+| Eval samples | 3,861 |
+| Train samples | 34,645 |
+
+**Hyperparameters** (from the sibling's Trainer-generated card):
+
+| | |
+|---|---|
+| Learning rate | 2e-05 |
+| Train / eval batch size | 32 / 32 |
+| Gradient accumulation | 2 (effective batch 64) |
+| Optimizer | AdamW (torch), β=(0.9, 0.999), ε=1e-08 |
+| LR schedule | linear, `warmup_steps: 0.06` |
+| Epochs | 100 |
+| Seed | 42 |
+| Train runtime | 54,567 s (~15.2 h) |
+| Total FLOPs | 2.99e18 |
+| Frameworks | Transformers 5.0.1.dev0, PyTorch 2.4.1+cu124, Datasets 4.5.0, Tokenizers 0.22.2 |
+
+The loss curve reported there runs from 1.4658 / 0.6942 accuracy at step 1,000 to 1.1143 / 0.7592 at
+step 52,000 — a long, flattening tail, with most of the gain landing in the first ~20k steps.
+
+> [!WARNING]
+> These numbers describe **MLM performance on the authors' own held-out split of an unidentified
+> corpus**. They are not a benchmark, they are not comparable to any published Bambara result, and
+> they say nothing about downstream task quality. **No downstream evaluation of this checkpoint
+> exists** — not on classification, NER, POS, QA, or retrieval. If you need to know whether it beats
+> stock XLM-R on your task, you have to measure it.
+
+## Limitations and risks
+
+- **Not a task model.** See the note at the top.
+- **Bambara only.** The additions target Bambara; the model retains stock XLM-R's 100-language
+  vocabulary, but the continued pretraining was Bambara-focused, and the general risk of catastrophic
+  forgetting on other languages applies. It has not been evaluated on any language other than
+  Bambara — including French, which co-occurs heavily with Bambara in Malian text.
+- **Orthography sensitivity.** Bambara orthography varies widely in practice (`ɛ` vs `è`, `ɔ` vs `o`,
+  tone marking, `nyɔgɔn` vs `ɲɔgɔn`). The added vocabulary was learned from one corpus distribution;
+  text written in a different convention will tokenize less efficiently. Note the tokenizer holds
+  *both* `▁Nyɔgon` and `▁Ŋɔgɔn`, which suggests the training corpus itself was orthographically mixed.
+- **No Unicode normalization** — see the tokenizer warning. Normalize to NFC yourself.
+- **512-token ceiling**, inherited from XLM-R.
+- **Untested for bias, toxicity, or factuality.** No such evaluation was run, by the authors or by me.
+  Inherits whatever is in CommonCrawl-scale XLM-R pretraining plus an unidentified Bambara corpus.
+- **F32 only.** No fp16/bf16 or quantized variant is published; the 1.12 GB `model.safetensors` is
+  full-precision.
+- **Mixed LayerNorm parameter naming in the checkpoint.** All 50 LayerNorms under `roberta.*` are
+  stored with the *legacy* `.gamma` / `.beta` names (e.g. `roberta.embeddings.LayerNorm.gamma`),
+  while `lm_head.layer_norm` uses the modern `.weight` / `.bias`. Stock
+  `FacebookAI/xlm-roberta-base` uses `.weight` / `.bias` throughout, so this was introduced by the
+  adaptation run, not inherited. It is transparent under `transformers` — 5.14.1 still ships the
+  `LayerNorm.gamma`→`weight` / `LayerNorm.beta`→`bias` rename in `conversion_mapping.py`, and
+  `AutoModelForMaskedLM.from_pretrained` loads all 202 tensors with **0 missing, 0 unexpected and 0
+  mismatched keys** — but that rename lives in a rule set explicitly labelled **`"legacy"`**.
+  **Tooling that reads `model.safetensors` directly** (custom loaders, ONNX/GGUF converters,
+  `safetensors` inspection scripts) **must expect the old names**, or it will silently leave every
+  encoder LayerNorm uninitialised.
+- **No pooler weights** — see the embeddings warning above. `pooler_output` is randomly initialised.
+
+## Undetermined
+
+Things I could not establish from the repo or any linked artifact. They are open questions, not
+omissions to be filled in with plausible guesses.
+
+- **Licence.** **No licence is declared anywhere** — not in the repo's card metadata, not in the Hub
+  tags, and not in any of the byte-identical sibling repos. Stock `FacebookAI/xlm-roberta-base` is
+  MIT, but a derivative's terms are the publisher's to state, and this one has not stated them.
+  **Treat the licensing status as unresolved and seek clarification from Djelia before reuse.**
+- **Training data identity.** "An 11M words Bambara dataset" is the entirety of what is documented.
+  The corpus, its sources, its licensing, its collection date, and its train/eval split are all
+  unknown. The 34,645 train / 3,861 eval sample counts are known; what a "sample" is (sentence?
+  document? 512-token block?) is not.
+- **How the 3,419 tokens were selected.** Whether by training a new SentencePiece/Unigram model on
+  Bambara and merging the delta, or by a frequency-threshold extraction — not documented. The
+  presence of a `##` token (a WordPiece convention) in an otherwise Unigram/Metaspace vocabulary, and
+  a stray `̸` (U+0338 combining long solidus overlay), both hint at an automated merge with light
+  filtering.
+- **How the new embeddings were initialised** before training (random, mean-of-subword, or otherwise).
+  I can show they are trained *now*; I cannot recover the starting point.
+- **The intermediate stages.** `oza75/xlm-roberta-bambara-cpt`, `-base`, `-initial`, `-base-01`, `-02`
+  form a chain, but which stage did the vocabulary extension and which did the continued pretraining
+  is not documented in any of them.
+- **Hardware.** Not recorded. Only the 54,567 s runtime and 2.99e18 FLOPs are known.
+- **Whether it improves any downstream task.** No downstream evaluation exists. The 30% tokenization
+  win is real and measured; a corresponding accuracy win is *plausible* but entirely unverified.
+- **Intended production role.** The repo carries no description of what Djelia built it for.
+
+## Provenance of this card
+
+Written by inspecting the repository directly rather than by restating its existing README. Every
+number above traces to one of:
+
+- the Hub API model record for `djelia/bm-xlm-roberta-base` (file list, blob sizes, LFS SHA-256s,
+  safetensors parameter count, commit history, timestamps);
+- `config.json` and `tokenizer_config.json` from this repo;
+- a direct comparison of `tokenizer.json` against `FacebookAI/xlm-roberta-base`;
+- the `model.safetensors` header and selected tensor byte-ranges (embedding rows and `lm_head.bias`),
+  fetched via HTTP range requests and analysed with NumPy;
+- a tokenization measurement I ran on a seeded 5,000-sentence sample of `djelia/bambara-texts`
+  (method stated in full above);
+- and, for the training section only, the Trainer-generated metadata in the byte-identical
+  `oza75/xlm-roberta-bambara-01`, explicitly labelled as second-hand.
+
+Every code snippet above was executed against this checkpoint on `transformers` 5.14.1 / `torch`
+2.13.0 before being included: the tokenizer loads cleanly (`mask_token='<mask>'`,
+`mask_token_id=253420`, `len(tokenizer)=253421`), `AutoModelForMaskedLM` loads all 202 tensors with
+zero missing/unexpected/mismatched keys, and the `fill-mask` outputs shown are the real ones. The
+missing pooler was found by diffing the tensor key set against stock XLM-R and then confirmed at
+runtime.
+
+Retrieval date: 2026-07-27. Nothing here is copied from an upstream card without verification, and
+no benchmark number appears that I did not either measure or trace to a named source file.