Add model card / README

README.md

@@ -0,0 +1,80 @@
+---
+license: other
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - gpt2
+  - childes
+  - child-directed-speech
+  - language-acquisition
+  - developmental
+  - sample-efficient-pretraining
+---
+
+# CHILDES-trained GPT-2 developmental ladder
+
+A ladder of **GPT-2-small models trained from scratch on child-directed speech (CHILDES)**
+at increasing amounts of input. It is the language-model side of a children-vs-LMs study of
+how vocabulary is acquired across "development," and is meant as a research artifact for work
+on developmental / sample-efficient language modeling.
+
+## What's here
+
+**180 models = 10 random seeds × 18 nested input budgets** (0.5M → 24M words).
+
+```
+development/seed{S}/rung{B}/
+  S ∈ {0, 1, 2, 3, 4, 5, 7, 42, 99, 123}
+  B ∈ {0.5M, 0.75M, 1M, 1.25M, 1.5M, 1.75M, 2M, 2.5M, 3M, 3.5M, 4M, 5M, 6M, 7M, 8M, 12M, 16M, 24M}
+```
+
+The budgets are **nested** (each larger budget contains the smaller), and a seed fixes both the
+initialization and the input-stream order — so each seed is an "individual" accumulating input,
+i.e. a developmental trajectory. Every model directory is self-contained (weights + tokenizer +
+config + `provenance.json`).
+
+## Load a model
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+sub = "development/seed42/rung24M"                     # pick any seed/budget
+model = AutoModelForCausalLM.from_pretrained("mcxfrank/childes-gpt2-ladder", subfolder=sub, torch_dtype="auto")
+tok   = AutoTokenizer.from_pretrained("mcxfrank/childes-gpt2-ladder", subfolder=sub)
+```
+
+Use `torch_dtype="auto"` to preserve the stored bf16 weights (otherwise they upcast to fp32).
+
+## Training
+
+- **Architecture:** GPT-2-small (125.8M params; 12 layers, 768-dim), trained **from scratch**.
+- **Tokenizer:** `GPT2_CHILDES` byte-pair encoding (52k vocab), trained on CHILDES (Feng et al. / TinyDialogues).
+- **Data:** English CHILDES (Feng et al. 2024 preparation). Budgets are counted in whitespace
+  words (the developmental currency); each seed shuffles conversations and takes cumulative
+  prefixes to each budget.
+- **Optimization:** up to 40 epochs with **early stopping** (patience 4) on held-out validation
+  loss; the lowest-validation-loss checkpoint is kept. AdamW, learning rate 1e-4 (linear decay,
+  no warmup), weight decay 0, batch size 8, sequence length 1024, no in-epoch shuffling.
+- **Precision:** stored as **bf16** safetensors.
+
+## Provenance & reproduction
+
+Each model carries a `provenance.json` (seed, budget, config). These weights were re-trained from
+the exact ladder samples used in the analysis; re-trained per-word surprisal reproduces the
+published values at **r = 0.988** (aggregate mean within ~0.04 nats). GPU nondeterminism makes
+them faithful re-trainings, not bit-identical.
+
+## Intended use & limitations
+
+Research artifacts for studying developmental / sample-efficient language modeling and word
+acquisition. These are **child-scale** models (≤24M words of input, English child-directed
+speech only) and are **not** intended for production use or as general-purpose language models.
+
+## Citation
+
+Paper: *Children, but not language models, learn words via accelerating accumulation* (Frank, in prep).
+Data & training pipeline: Feng et al. (2024); CHILDES (MacWhinney, TalkBank).
+
+## License
+
+**To be set by the repository owner.** The training data (CHILDES via TalkBank) is subject to
+TalkBank's terms of use; please respect those when using or redistributing these models.