Add model card

README.md

@@ -0,0 +1,84 @@
+# temporal-moe-extras
+
+Supporting artifacts for the Temporal-MoE experiments, 248 files, 39.78 GiB. These are things
+that are expensive to regenerate but are not themselves training checkpoints.
+
+## Layout
+
+```
+olmoe_adapt/
+  ckpt_bake_{CE,G,H,Er8,Er64,D}.pt   full optimizer and training state behind the router bakes
+  finetune_ids.pt                    tokenized finetune corpus token ids
+  bpb_slice_ids.pt                   bits-per-byte evaluation slice
+  cal2_gprime.pt                     calibration tensors
+  oseries/, oseries_ce/              paired evaluation slices, val.f16, idx.u8, paired masks
+  logs/                              bake, sweep, calibration and evaluation logs
+merged_ce_model/                     OLMoE backbone merged with the bake-CE router, loadable with
+                                     transformers, includes config and tokenizer
+run_captures/<run_name>/
+  router_log.pt                      per-run router traces, back the mechanistic interpretability
+                                     and residency result tables
+  delex_capture.pt                   delexicalization captures
+ablations/                           101 result tables as CSV, plus FINDINGS.md and README.md
+figures/                             43 figures
+tokenizer_tok16k/                    the 16k tokenizer, also mirrored in the corpus repository
+```
+
+## Notes
+
+`merged_ce_model/` is a derived artifact. It is included rather than regenerated because the merge is
+not guaranteed to be bit-reproducible.
+
+`ablations/` holds the result tables the paper draws on. `FINDINGS.md` and `README.md` in that
+directory are the written interpretation of those tables.
+
+The base OLMoE model is not mirrored here. It is public at
+[`allenai/OLMoE-1B-7B-0924`](https://huggingface.co/allenai/OLMoE-1B-7B-0924).
+
+## Training configuration
+
+Every run directory contains `run.meta` and `train.log`. `run.meta` records the full knob set for
+that run in two lines, for example:
+
+```
+[run] temporal_fine_g3_1e19 N=185766400 iters=17112 warmup=856 min_lr=0.0003 eval@1711
+[run] shape=s19opt H=800 L=14 ffn=4272 moe_ffn=184 grain=3 num_experts=192 shared_int=1100
+      dense=0 temporal=1 shared_mult=2 topk=18 heads=50 gb=256 mb=8 lr=3e-3 flops=1e19
+```
+
+`train.log` is the full Megatron stdout, including the complete argument namespace dumped at startup.
+Read these before drawing any conclusion from a checkpoint.
+
+## MANIFEST.csv and the `cited` column
+
+A manifest covering every file in all four repositories lives in the code repository. It has seven
+columns: `local_path`, `hf_repo`, `hf_path`, `bytes`, `sha256`, `run_name`, `cited`.
+
+The `cited` column marks whether a run is referenced by a results table in `results/ablations/*.csv`
+or by the paper:
+
+- `cited`, the run backs a published number. There are 58 of these.
+- `uncited`, the run is infrastructure validation, a smoke test, a throughput probe, or an aborted
+  run. It is kept for completeness and reproducibility, not because a table depends on it. There are
+  13 of these.
+- empty, the file is not scoped to a single run, for example a batch log or an evaluation output.
+
+Every `sha256` in the manifest was computed on this disk before upload and each file was verified to
+exist remotely with a matching byte size.
+
+## Links
+
+- Code: <https://github.com/ncylich/temporal-moe>
+- Paper: Temporal-MoE (short paper), see the `paper/` directory in the code repository
+- Upstream platform this work forks: [FLAME-MoE](https://github.com/cmu-flame/FLAME-MoE), [arXiv:2505.20225](https://arxiv.org/abs/2505.20225)
+
+## Companion repositories
+
+- [`ncylich/temporal-moe-ckpts`](https://huggingface.co/ncylich/temporal-moe-ckpts), Megatron training checkpoints
+- [`ncylich/temporal-moe-router-adapt`](https://huggingface.co/ncylich/temporal-moe-router-adapt), router adaptation safetensors
+- [`ncylich/temporal-moe-extras`](https://huggingface.co/ncylich/temporal-moe-extras), captures, merged model, result tables, figures
+- [`ncylich/temporal-moe-corpus`](https://huggingface.co/datasets/ncylich/temporal-moe-corpus), tokenized training corpus
+
+## Provenance
+
+Trained with a personal fork of FLAME-MoE. Not affiliated with or endorsed by the FLAME-MoE authors.