Buckets:
Encyclopaedia Britannica — Internet Archive scan mirror
A mirror of public-domain Encyclopaedia Britannica scan masters from the Internet Archive, covering editions 1st (1768–71) through 14th (1929) — everything out of copyright — from dozens of contributing libraries.
What's here
| Layer | Path | Contents |
|---|---|---|
| Data | ia/<ia-identifier>/<files> |
Scan masters (*_jp2.zip/.tar, *_images.zip, TIFF) + OCR layer (*_chocr.html.gz, *_djvu.xml, *_djvu.txt) + provenance (*_scandata.xml, *_meta.xml) |
| Per-item docs | ia/<ia-identifier>/README.md + _item.json |
What the item is: title, edition, year, contributor library, license, link back to IA |
| Catalog | _manifest/catalog.parquet / catalog.jsonl |
One row per file: path, size, md5, class, edition, contributor, license, IA URL |
| Worklists | _manifest/shard-{0..3}.jsonl |
The exact mirror worklist (used by the mirror jobs) |
| Progress | done/<ia-id>/<file>.json |
Per-file completion markers (size, md5, timing) |
| Reports | reports/shard-*.json |
Per-shard run summaries incl. any failures |
Scope decisions
- Mirrored: scan masters + one OCR layer. These are the irreplaceable bits.
- Not mirrored: IA's derived formats (PDF, EPUB, plain
hocr.html, abbyy) — regenerable from the masters if ever needed. This is ~1.9 TB deliberately skipped. - Not mirrored: anything
access-restrictedon IA (lending-only, mostly 15th edition era, 1931+). ~1.4 TB, legally unavailable for bulk download. - Not mirrored: 27 open-but-in-copyright or off-topic items (post-1930
yearbooks/printings, modern article extracts, books about Britannica).
8 of these were removed in a second pass (2026-08-25): the first screen keyed on
IA's
yearfield, and items with no year — Great Books of the Western World (1952) and Book of the Year 1938 among them — passed it unreviewed. Every blank-year item has since been checked by hand. Open status on IA ≠ public domain — some uploads carry uploader-applied CC licenses that don't bind the actual rightsholder. See_manifest/blocklist.jsonfor the full list with reasons. (US PD boundary as of 2026: published ≤ 1930.) - Deduped: exact duplicate files within IA items (e.g. NLS items carry each volume under both a long descriptive name and a short name) — the descriptive copy is kept. ~100 files / 19 GB.
Editions & sources
1,343 IA items. Major sets include:
- Editions 1–6 + supplements: National Library of Scotland scans
(
*EncyclopediaBritannicaNLS), plus independent copies from Getty Research Institute, Bavarian State Library, UC Libraries, National Library of Naples - Edition 7 (1842):
encyclopaedia-britannica-7ed-1842(NLS-sourced), BSB - Edition 8 (1853–60): Digital Library of India scans
- Edition 9 (1875–89, "Scholar's Edition"):
encyclopaedia-britannica-9ed-1875(complete, NLS-sourced) + DLI set - Edition 10 (1902–03):
encyclopaedia-britannica-10ed-1903(NLS-sourced) - Edition 11 (1910–11):
encyclopaedia-britannica-11th-and-12th-editionsplus Robarts/Toronto (*chisrich), DLI, andbwb_sets - Editions 12–14 (1922–1929): misc sets; 1929 entered US public domain Jan 2025, 1930 in Jan 2026. 1931+ printings are excluded (see blocklist)
Edition labels in the catalog ending in ? are inferred from publication
year only — verify against the item if it matters.
Licensing
All mirrored volumes were published 1768–1929 and are public domain by publication date in the US. Items with no IA year were reviewed by hand (see blocklist). Item-level
license metadata varies: some items carry an explicit Public Domain Mark or
CC-BY (e.g. the NLS items were uploaded by a third party with CC-BY-4.0, while
NLS's own data portal releases the same scans as public domain / CC0). Check
_item.json per item; when in doubt, attribute the contributing library.
Provenance & verification
- Source:
https://archive.org/details/<ia_id>per file (see catalog). - Every file was md5-verified against IA's own metadata during mirroring;
the expected md5 is in the catalog and in each
done/marker. - Mirrored with HF Jobs (cpu-basic), streaming IA → bucket, August 2026.
Mirror script + method: see
done/markers and run reports.
Rebuilding / resuming
The mirror is idempotent: rerunning a shard skips files already present with
matching size. To resume an interrupted shard, rerun the mirror job with the
same _manifest/shard-N.jsonl.
Related
NationalLibraryOfScotland/encyclopaedia-britannica-lance(dataset) — the NLS subset (eds 1–8, 155,388 pages) as per-page JPEG + ALTO XML in Lance format. That is the access layer; this bucket is the archival layer.
- Total size
- 3.97 TB
- Files
- 22,586
- Last updated
- Aug 26
- Pre-warmed CDN
- US EU US EU