docs: simplify card

README.md

@@ -12,476 +12,58 @@ tags:
 - bambara
 - bamanankan
 - mali
-- vocabulary-extension
 ---
 
 # bm-xlm-roberta-base
 
-A Bambara (Bamanankan, `bm`) adaptation of **XLM-RoBERTa base**, continued-pretrained with the
-masked-language-modelling objective and shipped with a **vocabulary-extended tokenizer** (253,421
-tokens vs. 250,002 in stock XLM-R).
+XLM-RoBERTa base adapted to Bambara (Bamanankan, `bm`): continued masked-language-model
+pretraining with a vocabulary-extended tokenizer that adds Bambara words and the characters
+`Ɛ Ɔ Ɲ Ŋ`.
 
-> [!IMPORTANT]
-> **This is a masked-LM checkpoint, not a task model.** The architecture is
-> `XLMRobertaForMaskedLM`. It predicts `<mask>` and produces contextual representations. It does
-> **not** classify, tag, translate, or embed sentences out of the box. Use it as an *initialisation*
-> for fine-tuning on sequence classification, token classification (NER/POS), extractive QA, or as
-> the encoder backbone of a retrieval model.
-
-The headline reason to prefer this over stock `FacebookAI/xlm-roberta-base` for Bambara is
-tokenization: on a 5,000-sentence Bambara sample it produces **30.0% fewer tokens** (measurement and
-method below), which directly translates to shorter sequences, more real context inside the 512-token
-window, and cheaper fine-tuning.
-
-## Model details
-
-| | |
-|---|---|
-| Repository | `djelia/bm-xlm-roberta-base` (public) |
-| Architecture | `XLMRobertaForMaskedLM` (encoder-only, absolute position embeddings) |
-| Parameters | 280,924,397 (all `F32`) |
-| Layers / hidden / heads / FFN | 12 / 768 / 12 / 3072 |
-| Activation | `gelu` |
-| `max_position_embeddings` | 514 (usable sequence length **512**; `model_max_length: 512`) |
-| `vocab_size` | **253,421** |
-| `type_vocab_size` | 1 |
-| `tie_word_embeddings` | `true` (LM head decoder is tied to the input embedding matrix) |
-| Special token ids | `<s>`=0, `<pad>`=1, `</s>`=2, `<unk>`=3, **`<mask>`=253420** |
-| Files | `config.json`, `model.safetensors` (1,123,721,924 B), `tokenizer.json` (17,001,775 B), `tokenizer_config.json` |
-| Total repo storage | 1,140,723,699 B (~1.14 GB) |
-| Created / last modified | 2026-02-04 (both; all four commits authored by `oza75`) |
-| `transformers_version` in config | `5.0.0` |
-
-All of the above is read from the repo's `config.json`, `tokenizer_config.json`, and the Hub API
-model record.
-
-### Lineage
-
-The repo is tagged `base_model: FacebookAI/xlm-roberta-base`, which is the ultimate ancestor: the
-first 250,001 vocabulary entries and their ids are inherited unchanged from stock XLM-R (verified by
-an ordered vocabulary comparison, see below).
-
-The immediate provenance is a training run in the author's personal namespace. The
-`model.safetensors` in this repo is **byte-identical** (SHA-256
-`025cd2563f9aa71c26999eed79c0115b400ce29bbe033a9026312032a2bbd9c2`) to the one in
-[`oza75/xlm-roberta-bambara-01`](https://huggingface.co/oza75/xlm-roberta-bambara-01). I compared the
-LFS SHA-256 of `model.safetensors` across every `oza75/*xlm-roberta*bambara*` repo; that one matched
-and the other six did not. This repo was uploaded first (2026-02-04); the `oza75` copy was published
-later (2026-04-23) *with* the Trainer-generated metadata that this repo omits.
-
-The sibling repo declares its own base as `oza75/xlm-roberta-bambara-base-01`, which in turn declares
-`oza75/xlm-roberta-bambara-initial` — so the real chain is a multi-stage adaptation
-(`FacebookAI/xlm-roberta-base` → vocabulary extension + continued pretraining stages → this
-checkpoint), not a single fine-tune. So this is **not** a re-publication of a different model: it is
-the same artifact as `oza75/xlm-roberta-bambara-01`, published to the org namespace first.
-
-## Tokenizer: what the 3,419 added tokens actually are
-
-This is the most substantive difference from stock XLM-R, so it is worth stating precisely. I
-downloaded both `tokenizer.json` files and compared them directly.
-
-**Structure of the change.** The stock 250,002-entry vocabulary is preserved *in its original order*
-at ids 0–250,000. The 3,419 new tokens were **appended** at ids 250,001–253,419, and stock's `<mask>`
-was **moved from id 250,001 to id 253,420** (the only ordered mismatch in the overlapping range). No
-stock token was removed.
-
-> [!WARNING]
-> **Token ids are not interchangeable with stock XLM-R.** Ids 0–250,000 match, but `<mask>` moved
-> from 250001 → 253420. Never mix this tokenizer with a stock XLM-R checkpoint or vice versa. Always
-> load the tokenizer from this repo.
-
-**Composition of the 3,419 added tokens** (my count over `tokenizer.json`):
-
-| Property | Count | Share |
-|---|---|---|
-| Word-initial (begin with the `▁` metaspace marker) | 2,541 | 74.3% |
-| Contain a Bambara-specific letter (`ɛ ɔ ɲ ŋ` / `Ɛ Ɔ Ɲ Ŋ`) | 1,333 | 39.0% |
-| Pure ASCII-Latin (e.g. `▁taa`, `▁jama`, `▁kelen`, `baara`, `muso`) | 2,042 | 59.7% |
-| Punctuation-bearing (`▁,`, `▁.`, `▁?..`, `▁....`, and the odd `##`) | 38 | 1.1% |
-| Digit-bearing | 0 | 0% |
-| Single-character (`ԑ`, `Ɛ`, `Ŋ`, `Ɲ`, and a stray `̸`) | 5 | 0.1% |
-
-Token length (stripped of `▁`) peaks at 4–5 characters; the additions are overwhelmingly Bambara
-*words and morphemes*, not characters. Representative examples: `▁bɛ`, `▁tɛ`, `▁kɛ`, `▁bɛɛ`, `▁fɔ`,
-`▁mɔgɔ`, `▁jama`, `▁kelen`, `▁yɔrɔ`, `▁cogo`, `▁baara`, `▁muso`, plus productive suffix pieces
-(`kɔrɔ`, `tigi`, `ya`, `nba`) and Malian proper nouns (`▁Ɲafunke`, `▁Ɔsitarali`, `▁Ɛsipaɲi`).
-
-**So it is both.** The additions are dominated by Bambara whole words (which stock XLM-R has no
-entries for at all), and they *also* close a genuine character-coverage hole:
-
-> The capital letters **`Ɛ` (U+0190), `Ɲ` (U+01F8), and `Ŋ` (U+014A) are absent from the stock
-> XLM-R vocabulary entirely** — I confirmed that stock encodes each of them to `<unk>` (id 3).
-> Lowercase `ɛ ɔ ɲ ŋ` and capital `Ɔ` are present in stock, but only as isolated single characters,
-> so stock shreds every Bambara word containing them.
-
-A concrete side-by-side (`add_special_tokens=True`):
-
-```
-Input: "Mɔgɔw bɛɛ bɛ bange hɔrɔnya ni danbe ni josira lakanani na."
-
-this repo (20 tokens):
-  <s> ▁Mɔgɔ w ▁bɛɛ ▁bɛ ▁bange ▁hɔrɔn ya ▁ni ▁dan be ▁ni ▁jo sira ▁laka na ni ▁na . </s>
-
-stock XLM-R (30 tokens):
-  <s> ▁M ɔ g ɔ w ▁b ɛ ɛ ▁b ɛ ▁bang e ▁h ɔ r ɔ nya ▁ni ▁dan be ▁ni ▁jos ira ▁la kan ani ▁na . </s>
-```
-
-### Measured tokenization benefit
-
-**This is my own measurement, not a figure taken from any published card.** Method, stated so it is
-reproducible:
-
-- **Corpus:** [`djelia/bambara-texts`](https://huggingface.co/datasets/djelia/bambara-texts),
-  file `data/train-00000-of-00001.parquet`, column `text` (84,328 rows).
-- **Sample:** rows with non-null, non-empty `text`, then
-  `pandas.Series.sample(n=5000, random_state=0)` — a fixed seed, so the exact sample is reproducible.
-- **Tokenizers:** `tokenizers.Tokenizer.from_file()` on `tokenizer.json` from this repo and from
-  `FacebookAI/xlm-roberta-base`, `encode_batch(..., add_special_tokens=False)`.
-- **Sample size:** 4,614,792 characters / 917,625 whitespace-delimited words.
-
-| Metric | `djelia/bm-xlm-roberta-base` | stock `xlm-roberta-base` |
-|---|---|---|
-| Total tokens | **1,454,892** | 2,079,639 |
-| Tokens per word | **1.585** | 2.266 |
-| Characters per token | **3.172** | 2.219 |
-| `<unk>` tokens (id 3) | **48** (0.0033%) | 1,382 (0.0665%) |
-| Sentences containing ≥1 `<unk>` | **9** / 5,000 | 556 / 5,000 |
-
-**Corpus-level reduction: 30.04% fewer tokens.** Per sentence, the reduction is a median of 30.00%
-(mean 28.78%, p10 20.72%, p90 35.43%), and **this tokenizer was shorter-or-equal on 100.0% of the
-5,000 sentences** — there is no regression tail.
-
-A useful sanity check on whether the new vocabulary is actually load-bearing: **29.38% of all tokens
-produced on this Bambara corpus come from the 3,419 added ids.** 1.3% of the vocabulary carries
-nearly 30% of the token mass.
-
-> [!NOTE]
-> The corpus is a mixed-provenance Bambara collection (largest sources: `saillab/alpaca-bambara-cleaned`,
-> `bigscience/xP3all`, `cis-lmu/GlotCC-V1`, `facebook/belebele`). The 30% figure describes *this*
-> distribution of Bambara text. Orthographically non-standard or heavily code-switched Bambara will
-> land differently.
-
-### Tokenizer caveats
-
-> [!WARNING]
-> **The Unicode normalizer was replaced, and it no longer normalizes Unicode.** Stock XLM-R uses
-> SentencePiece's `Precompiled` charsmap (NFKC-style folding). This tokenizer instead uses
-> `Sequence[Strip(right), Replace(" {2,}" → "▁")]`. Two observed consequences:
->
-> - **NFC vs. NFD input tokenizes differently.** `"ka nà"` in NFC → `▁n`, `à`; the same string in
->   NFD → `▁na`, `̀`. Stock produces the identical result for both. **Normalize your text to NFC
->   yourself** before tokenizing, at both fine-tuning and inference time, or you will get silent
->   train/serve skew.
-> - **Compatibility characters are not folded.** `①` and `fi` pass through as-is (stock maps them to
->   `1` and `fi`).
-
-> [!WARNING]
-> `tokenizer_config.json` declares `"tokenizer_class": "XLMRobertaTokenizer"` and `"use_fast": false`,
-> but the repo ships **only `tokenizer.json`** — there is no `sentencepiece.bpe.model`. I verified the
-> tokenizer loads correctly on **`transformers` 5.14.1**. On older `transformers` 4.x, where
-> `use_fast: false` would attempt to build the SentencePiece-backed slow tokenizer, loading may fail;
-> pass `use_fast=True` explicitly if you hit that.
-
-## Are the new embeddings actually trained?
-
-Yes. This matters — a vocabulary-extended checkpoint whose new rows were never trained is worthless —
-so I checked it rather than assuming.
-
-The embedding matrix **was resized correctly**: `roberta.embeddings.word_embeddings.weight` has shape
-`[253421, 768]`, read from the `model.safetensors` header. Four independent pieces of evidence say the
-added rows carry learned signal:
-
-1. **Magnitude is inconsistent with fresh initialisation.** Rows 250,001–253,420 have element-wise
-   σ = 0.1157 and mean L2 norm 3.204 (σ 0.273). `config.json` sets `initializer_range: 0.02`, so a
-   freshly-initialised row would have norm ≈ 0.02·√768 ≈ **0.55** — off by ~6×.
-2. **Norms track token frequency.** Correlation between a token's unigram score in `tokenizer.json`
-   and its embedding norm is **r = 0.818** over the added ids (vs. r = 0.277 over the stock ids).
-   Randomly initialised rows cannot correlate with corpus frequency.
-3. **The LM head bias for the new ids is non-zero and varied**: over ids 250,001+, mean −0.011,
-   σ 0.122, range [−2.644, 0.347], with **zero** exactly-zero entries.
-4. **Nearest neighbours are semantically coherent** (cosine similarity over the full L2-normalised
-   embedding matrix):
-
-   | Token | Top neighbours |
-   |---|---|
-   | `▁bɛ` ("is") | `▁tɛ` (0.75) — *its own negative form* — `▁Bɛ`, `bɛ`, `▁Nbɛ` |
-   | `▁mɔgɔ` ("person") | `▁Mogo`, `mɔgɔ`, `▁muso` ("woman"), `▁Krɛcɛn` |
-   | `▁yɔrɔ` ("place") | `▁Cɔgɔ`/`▁cogo` ("manner"), `▁waati` ("time"), `▁Yɔrɔ` |
-   | `▁kelen` ("one") | `kelen`, `▁Saaba` ("three"), `▁wɔɔrɔ` ("six"), `▁Kelen` |
-   | `▁Ɲafunke` (Malian town) | a tight cluster of other Malian toponyms at 0.79–0.83 — `▁Direyi`, `▁Yuwaru`, `▁Lere`, `▁Duwanza`, `▁Kemasina`, `▁Burɛmu` |
-
-   Numerals cluster with numerals, toponyms with toponyms, and a copula with its own negation. That
-   structure cannot arise from noise.
-
-> [!NOTE]
-> One observation worth recording without over-reading it: the added rows sit at a systematically
-> **lower norm than the stock rows** (mean 3.204 vs. 5.505) and are much tighter (σ 0.273 vs. 0.432).
-> This is the expected signature of embeddings that received fewer gradient updates than the
-> pretrained ones — they are trained, but for less wall-clock than the original XLM-R vocabulary. No
-> action is implied; it is simply the shape of the checkpoint.
+This is a masked-LM checkpoint, not a task model. It predicts `<mask>` and produces
+contextual representations; use it as the initialisation for fine-tuning on classification,
+token tagging, QA, or as a retrieval encoder.
 
 ## Usage
 
-### Fill-mask
-
-XLM-R's mask token is **`<mask>`** (not `[MASK]`). In this checkpoint its id is **253420**.
+The mask token is `<mask>`, not `[MASK]`.
 
 ```python
 from transformers import pipeline
 
 fill = pipeline("fill-mask", model="djelia/bm-xlm-roberta-base")
 
-# "All people are born in freedom __ dignity."  (expected fill: "ni" = "and")
 for p in fill("Mɔgɔw bɛɛ bɛ bange hɔrɔnya <mask> danbe la."):
-    print(f"{p['score']:.4f}  {p['token_str']!r}  ->  {p['sequence']}")
+    print(f"{p['score']:.4f}  {p['token_str']!r}")
 ```
 
-Or explicitly, without the pipeline:
+Always load the tokenizer from this repo — token ids are not interchangeable with stock
+XLM-R.
 
-```python
-import torch
-from transformers import AutoTokenizer, AutoModelForMaskedLM
-
-model_id = "djelia/bm-xlm-roberta-base"
-tokenizer = AutoTokenizer.from_pretrained(model_id)
-model = AutoModelForMaskedLM.from_pretrained(model_id).eval()
-
-text = "Mɔgɔw bɛɛ bɛ bange hɔrɔnya <mask> danbe la."
-inputs = tokenizer(text, return_tensors="pt")
+## Architecture
 
-with torch.no_grad():
-    logits = model(**inputs).logits
-
-mask_index = (inputs.input_ids == tokenizer.mask_token_id).nonzero()[0, 1]
-top = logits[0, mask_index].topk(5)
-
-for score, token_id in zip(top.values.tolist(), top.indices.tolist()):
-    print(f"{score:7.3f}  {tokenizer.decode([token_id])!r}")
-```
-
-Both snippets were executed against this checkpoint (`transformers` 5.14.1, `torch` 2.13.0). The
-actual output for that sentence — the opening of the UDHR Article 1 in Bambara, *"every person is
-born in freedom \_\_ dignity"* — is:
-
-| Rank | Token | Probability |
-|---|---|---|
-| 1 | `ni` ("and") | 0.875 |
-| 2 | `ani` ("and", alt. form) | 0.097 |
-| 3 | `,` | 0.004 |
-| 4 | `kɔnɔ` ("in/inside") | 0.003 |
-| 5 | `la` (locative postposition) | 0.003 |
+| | |
+| --- | --- |
+| Class | `XLMRobertaForMaskedLM` (encoder-only) |
+| Parameters | 280,924,397 (`F32`) |
+| Layers / hidden / heads / FFN | 12 / 768 / 12 / 3072 |
+| Max sequence length | 512 |
+| `vocab_size` | 253,421 |
+| Special tokens | `<s>`=0, `<pad>`=1, `</s>`=2, `<unk>`=3, `<mask>`=253420 |
 
-The top two predictions are both correct Bambara conjunctions, and rank 1 is the word actually used
-in the source text — a sanity check that the checkpoint is intact and the mask token is right, not a
-benchmark.
+## Notes
 
-### Contextual embeddings
+The checkpoint carries no pooler weights, so `pooler_output` from `AutoModel` is randomly
+initialised. Take `last_hidden_state` and pool it yourself:
 
 ```python
-import torch
-from transformers import AutoTokenizer, AutoModel
+from transformers import AutoModel, AutoTokenizer
 
 tokenizer = AutoTokenizer.from_pretrained("djelia/bm-xlm-roberta-base")
 encoder = AutoModel.from_pretrained("djelia/bm-xlm-roberta-base").eval()
 
-batch = tokenizer(["Aw ni ce.", "I ka kɛnɛ wa?"], padding=True, return_tensors="pt")
-with torch.no_grad():
-    hidden = encoder(**batch).last_hidden_state  # (batch, seq_len, 768)
-```
-
-> [!WARNING]
-> **Do not use `pooler_output`.** This checkpoint contains **no pooler weights**. I diffed its tensor
-> key set against stock `FacebookAI/xlm-roberta-base` (after normalising the `gamma`/`beta` naming
-> below) and the *only* difference is that `roberta.pooler.dense.weight` and
-> `roberta.pooler.dense.bias` are **absent here**. Confirmed at runtime: `AutoModel.from_pretrained`
-> reports `missing_keys = {'pooler.dense.weight', 'pooler.dense.bias'}`, so those two tensors are
-> **randomly initialised** and `pooler_output` is meaningless noise. Use `last_hidden_state` and pool
-> it yourself. (`AutoModel` also reports the five `lm_head.*` tensors as unexpected — that is normal
-> and harmless when loading an MLM checkpoint into a bare encoder.)
-
-> [!WARNING]
-> `last_hidden_state` gives *token* representations. Mean-pooling them is **not** a sentence-embedding
-> model — no contrastive objective was ever applied here. For retrieval, fine-tune with a
-> sentence-similarity objective (e.g. `sentence-transformers`) on top of this encoder.
-
-### Fine-tuning starting point
-
-Swap in whichever head your task needs; the encoder weights load, the head is initialised fresh.
-
-```python
-from transformers import (
-    AutoModelForSequenceClassification,  # or AutoModelForTokenClassification
-    AutoTokenizer,
-    DataCollatorWithPadding,
-    Trainer,
-    TrainingArguments,
-)
-
-MODEL_ID = "djelia/bm-xlm-roberta-base"
-
-tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
-model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID, num_labels=3)
-
-def preprocess(batch):
-    # Normalize to NFC first — this tokenizer does no Unicode normalization of its own.
-    import unicodedata
-    texts = [unicodedata.normalize("NFC", t) for t in batch["text"]]
-    return tokenizer(texts, truncation=True, max_length=512)
-
-tokenized = your_dataset.map(preprocess, batched=True)
-
-trainer = Trainer(
-    model=model,
-    args=TrainingArguments(
-        output_dir="bm-xlmr-finetuned",
-        learning_rate=2e-5,          # standard XLM-R base range: 1e-5 .. 3e-5
-        per_device_train_batch_size=16,
-        num_train_epochs=3,
-        warmup_ratio=0.06,
-        weight_decay=0.01,
-        fp16=True,
-    ),
-    train_dataset=tokenized["train"],
-    eval_dataset=tokenized["validation"],
-    data_collator=DataCollatorWithPadding(tokenizer),
-)
-trainer.train()
+hidden = encoder(**tokenizer(["Aw ni ce."], return_tensors="pt")).last_hidden_state
 ```
 
-Two practical notes:
-
-- **Do not resize the embeddings** unless you are adding your own special tokens. The matrix is
-  already 253,421 rows and matches the shipped tokenizer.
-- Because the added embeddings are less-trained than the inherited ones (see above), leaving the
-  embedding matrix **unfrozen** during fine-tuning is the safer default.
-
-## Training
-
-> [!NOTE]
-> This repo ships **no** training artifacts (no `trainer_state.json`, `training_args.bin`, or
-> `all_results.json`). Everything in this section is read from
-> [`oza75/xlm-roberta-bambara-01`](https://huggingface.co/oza75/xlm-roberta-bambara-01), whose
-> `model.safetensors` is byte-identical to this one (SHA-256 verified, see *Lineage*). It is therefore
-> the same run — but it is second-hand for *this* repo, and is labelled as such.
-
-The original card in this repo states the model was trained "on an 11M words Bambara dataset"
-(author's claim, preserved verbatim; the dataset itself is not identified).
-
-**Reported evaluation results** (MLM objective, held-out split of the same unnamed corpus):
-
-| Metric | Value |
-|---|---|
-| Eval loss | 1.1160 |
-| Eval masked-token accuracy | 0.7599 |
-| Eval perplexity | 3.0525 |
-| Eval samples | 3,861 |
-| Train samples | 34,645 |
-
-**Hyperparameters** (from the sibling's Trainer-generated card):
-
-| | |
-|---|---|
-| Learning rate | 2e-05 |
-| Train / eval batch size | 32 / 32 |
-| Gradient accumulation | 2 (effective batch 64) |
-| Optimizer | AdamW (torch), β=(0.9, 0.999), ε=1e-08 |
-| LR schedule | linear, `warmup_steps: 0.06` |
-| Epochs | 100 |
-| Seed | 42 |
-| Train runtime | 54,567 s (~15.2 h) |
-| Total FLOPs | 2.99e18 |
-| Frameworks | Transformers 5.0.1.dev0, PyTorch 2.4.1+cu124, Datasets 4.5.0, Tokenizers 0.22.2 |
-
-The loss curve reported there runs from 1.4658 / 0.6942 accuracy at step 1,000 to 1.1143 / 0.7592 at
-step 52,000 — a long, flattening tail, with most of the gain landing in the first ~20k steps.
-
-> [!WARNING]
-> These numbers describe **MLM performance on the authors' own held-out split of an unidentified
-> corpus**. They are not a benchmark, they are not comparable to any published Bambara result, and
-> they say nothing about downstream task quality. **No downstream evaluation of this checkpoint
-> exists** — not on classification, NER, POS, QA, or retrieval. If you need to know whether it beats
-> stock XLM-R on your task, you have to measure it.
-
-## Limitations and risks
-
-- **Not a task model.** See the note at the top.
-- **Bambara only.** The additions target Bambara; the model retains stock XLM-R's 100-language
-  vocabulary, but the continued pretraining was Bambara-focused, and the general risk of catastrophic
-  forgetting on other languages applies. It has not been evaluated on any language other than
-  Bambara — including French, which co-occurs heavily with Bambara in Malian text.
-- **Orthography sensitivity.** Bambara orthography varies widely in practice (`ɛ` vs `è`, `ɔ` vs `o`,
-  tone marking, `nyɔgɔn` vs `ɲɔgɔn`). The added vocabulary was learned from one corpus distribution;
-  text written in a different convention will tokenize less efficiently. Note the tokenizer holds
-  *both* `▁Nyɔgon` and `▁Ŋɔgɔn`, which suggests the training corpus itself was orthographically mixed.
-- **No Unicode normalization** — see the tokenizer warning. Normalize to NFC yourself.
-- **512-token ceiling**, inherited from XLM-R.
-- **Untested for bias, toxicity, or factuality.** No such evaluation was run, by the authors or by me.
-  Inherits whatever is in CommonCrawl-scale XLM-R pretraining plus an unidentified Bambara corpus.
-- **F32 only.** No fp16/bf16 or quantized variant is published; the 1.12 GB `model.safetensors` is
-  full-precision.
-- **Mixed LayerNorm parameter naming in the checkpoint.** All 50 LayerNorms under `roberta.*` are
-  stored with the *legacy* `.gamma` / `.beta` names (e.g. `roberta.embeddings.LayerNorm.gamma`),
-  while `lm_head.layer_norm` uses the modern `.weight` / `.bias`. Stock
-  `FacebookAI/xlm-roberta-base` uses `.weight` / `.bias` throughout, so this was introduced by the
-  adaptation run, not inherited. It is transparent under `transformers` — 5.14.1 still ships the
-  `LayerNorm.gamma`→`weight` / `LayerNorm.beta`→`bias` rename in `conversion_mapping.py`, and
-  `AutoModelForMaskedLM.from_pretrained` loads all 202 tensors with **0 missing, 0 unexpected and 0
-  mismatched keys** — but that rename lives in a rule set explicitly labelled **`"legacy"`**.
-  **Tooling that reads `model.safetensors` directly** (custom loaders, ONNX/GGUF converters,
-  `safetensors` inspection scripts) **must expect the old names**, or it will silently leave every
-  encoder LayerNorm uninitialised.
-- **No pooler weights** — see the embeddings warning above. `pooler_output` is randomly initialised.
-
-## Undetermined
-
-Things I could not establish from the repo or any linked artifact. They are open questions, not
-omissions to be filled in with plausible guesses.
-
-- **Licence.** **No licence is declared anywhere** — not in the repo's card metadata, not in the Hub
-  tags, and not in any of the byte-identical sibling repos. Stock `FacebookAI/xlm-roberta-base` is
-  MIT, but a derivative's terms are the publisher's to state, and this one has not stated them.
-  **Treat the licensing status as unresolved and seek clarification from Djelia before reuse.**
-- **Training data identity.** "An 11M words Bambara dataset" is the entirety of what is documented.
-  The corpus, its sources, its licensing, its collection date, and its train/eval split are all
-  unknown. The 34,645 train / 3,861 eval sample counts are known; what a "sample" is (sentence?
-  document? 512-token block?) is not.
-- **How the 3,419 tokens were selected.** Whether by training a new SentencePiece/Unigram model on
-  Bambara and merging the delta, or by a frequency-threshold extraction — not documented. The
-  presence of a `##` token (a WordPiece convention) in an otherwise Unigram/Metaspace vocabulary, and
-  a stray `̸` (U+0338 combining long solidus overlay), both hint at an automated merge with light
-  filtering.
-- **How the new embeddings were initialised** before training (random, mean-of-subword, or otherwise).
-  I can show they are trained *now*; I cannot recover the starting point.
-- **The intermediate stages.** `oza75/xlm-roberta-bambara-cpt`, `-base`, `-initial`, `-base-01`, `-02`
-  form a chain, but which stage did the vocabulary extension and which did the continued pretraining
-  is not documented in any of them.
-- **Hardware.** Not recorded. Only the 54,567 s runtime and 2.99e18 FLOPs are known.
-- **Whether it improves any downstream task.** No downstream evaluation exists. The 30% tokenization
-  win is real and measured; a corresponding accuracy win is *plausible* but entirely unverified.
-- **Intended production role.** The repo carries no description of what Djelia built it for.
-
-## Provenance of this card
-
-Written by inspecting the repository directly rather than by restating its existing README. Every
-number above traces to one of:
-
-- the Hub API model record for `djelia/bm-xlm-roberta-base` (file list, blob sizes, LFS SHA-256s,
-  safetensors parameter count, commit history, timestamps);
-- `config.json` and `tokenizer_config.json` from this repo;
-- a direct comparison of `tokenizer.json` against `FacebookAI/xlm-roberta-base`;
-- the `model.safetensors` header and selected tensor byte-ranges (embedding rows and `lm_head.bias`),
-  fetched via HTTP range requests and analysed with NumPy;
-- a tokenization measurement I ran on a seeded 5,000-sentence sample of `djelia/bambara-texts`
-  (method stated in full above);
-- and, for the training section only, the Trainer-generated metadata in the byte-identical
-  `oza75/xlm-roberta-bambara-01`, explicitly labelled as second-hand.
-
-Every code snippet above was executed against this checkpoint on `transformers` 5.14.1 / `torch`
-2.13.0 before being included: the tokenizer loads cleanly (`mask_token='<mask>'`,
-`mask_token_id=253420`, `len(tokenizer)=253421`), `AutoModelForMaskedLM` loads all 202 tensors with
-zero missing/unexpected/mismatched keys, and the `fill-mask` outputs shown are the real ones. The
-missing pooler was found by diffing the tensor key set against stock XLM-R and then confirmed at
-runtime.
-
-Retrieval date: 2026-07-27. Nothing here is copied from an upstream card without verification, and
-no benchmark number appears that I did not either measure or trace to a named source file.
+Mean-pooled token vectors are not sentence embeddings — no contrastive objective was applied
+here. Fine-tune with a sentence-similarity objective if you need retrieval.