Simpler model card; source1.py comment wording

EVALUATION.md

@@ -179,7 +179,7 @@ rubric text. The exams were graded with an older revision of the rubric, from be
 ads.
 
 The grader returns the 13 fields only. Its reference overall score and keep flag are computed from its scores in the
-same way as Source-1's: the overall formula of the rubric (see [README.md](README.md#what-it-returns)), and keep =
+same way as Source-1's: the overall formula of the rubric (see [README.md](README.md#what-you-get)), and keep =
 false when the rubric's hard filters match (`toxicity >= 4`, `spam_seo >= 4` or `boilerplate >= 4.5`). "The grader's
 drops" in this file are those chunks: spam, boilerplate or toxic text under the hard filters.
 
@@ -792,7 +792,7 @@ on a single field; 4 labels and 1 keep decision changed).
   document for text-and-data-mining and AI-training reservations. This screening did not catch notices worded in ways
   the patterns miss, or reservations made outside the text itself (on the terms pages of sites the rules do not list,
   or in machine-readable opt-out signals such as robots.txt). If you find such a document, tell us (see the contact
-  section of [README.md](README.md#contact-and-takedown)).
+  section of [README.md](README.md#contact)).
 - **No reproduction kit.** See [Reproducing the evaluation](#reproducing-the-evaluation).
 
 </details>

README.md

@@ -6,6 +6,7 @@ base_model_relation: finetune
 # "library_name: transformers" entry would offer AutoModel and pipeline snippets that load the backbone without
 # the trained heads and return meaningless scores.
 pipeline_tag: text-classification
+inference: false
 language:
 - en
 - ar
@@ -136,33 +137,30 @@ datasets:
 
 # Source-1
 
-Source-1 scores text as pretraining data for language models. It reads up to 8,192 tokens at a time, in 53 languages.
-For each chunk it returns 13 fields (labels, quality scores and red flags), an overall 0-5 score and a keep/drop
-decision. 307M parameters, Apache-2.0.
+Source-1 scores how useful a text is for training language models. It returns 13 scores and labels, such as
+educational value, spam, toxicity and topic, plus one overall score and a keep-or-drop decision. It reads 53
+languages, up to 8,192 tokens at a time. It has 307M parameters and is free to use under Apache-2.0.
 
 ![Source-1 vs. public quality scorers](images/source1_benchmark.png)
 
 ## Highlights
 
-- **Outperforms every public quality scorer we tested, on all three test sets**, measured as agreement with an
-  independent LLM grader using Source-1's rubric. On the held-out set: 0.90 against 0.76 for propella-1 4B; on its 159
-  English chunks, 0.86 against 0.53 for the FineWeb-Edu classifier.
-- **About 13x fewer parameters than propella-1 4B, and still closer to the grader.** Source-1 has 307M parameters,
-  propella-1 4B about 4.0B, and Source-1 agrees with the grader more closely on all three sets.
-- **Within 0.012 of its 27B teacher at about 1/88 the size.** On the held-out set Source-1 scores 0.900; the
-  open-weight 27B LLM teacher it learned from scores 0.912.
-- **Ahead on educational value alone, too.** On the English exam its `educational_value` reaches 0.90 rank agreement
-  with the grader's, against 0.61 for the FineWeb-Edu classifier and 0.88 for propella-1 4B.
+The numbers are rank agreement with an independent proprietary LLM grader: how closely a model puts texts in the same
+order as the grader does (1.0 means the same order, 0 means no link).
 
-*Measured as agreement with an independent LLM grader applying Source-1's own rubric, on held-out data from the same
-kinds of sources; see [EVALUATION.md](EVALUATION.md) for methods, intervals and limits.*
+- **Beats all 16 public quality scorers we tested, including FineWeb-Edu and propella-1, on all three test sets.** On the main test set (495 held-out
+  texts in 53 languages): 0.90 vs 0.76 for the best of them, propella-1 4B, a model 13x larger.
+- **Nearly matches its teacher**, the open-weight 27B LLM that labeled its training data, at 1/88 the size: 0.90 vs 0.91
+  on the main test set.
+- **Also ahead when judged on educational value alone**, the thing most public scorers were built for.
+
+*Caveat: the grader scored with Source-1's own rubric (its scoring guide), so this test plays to Source-1's
+strengths. See [Limitations](#limitations) and [EVALUATION.md](EVALUATION.md).*
 
 ## Quick start
 
-Source-1 runs through the included `source1.py`, which needs only `torch`, `transformers`, `safetensors` and
-`tokenizers` (no `trust_remote_code`). Do not load it with transformers' `pipeline` or `AutoModel` classes: they load
-the backbone without the trained heads in `heads.safetensors` and return meaningless scores. The model is published on
-the Hugging Face Hub as `msmth/Source-1`.
+Source-1 runs through the included `source1.py`. Do not load it with transformers' `pipeline` or `AutoModel`: they
+skip the trained scoring heads and give meaningless scores.
 
 ```bash
 pip install -U huggingface_hub            # provides the hf command
@@ -175,6 +173,20 @@ python source1.py --model . --input examples/sample.jsonl --output scores.jsonl
 ```python
 from source1 import Source1
 
+model = Source1.from_pretrained(".")
+doc = model.score("Photosynthesis is how plants turn light, water and carbon dioxide into sugar and oxygen.")
+print(doc["overall"], doc["keep"])   # a 0-5 score and the keep/drop decision
+```
+
+<details>
+<summary>More usage: many texts, precision, options, speed, files</summary>
+
+`source1.py` needs only `torch`, `transformers`, `safetensors` and `tokenizers` (no `trust_remote_code`). The model
+is published on the Hugging Face Hub as `msmth/Source-1`.
+
+```python
+from source1 import Source1
+
 model = Source1.from_pretrained(".")          # a local directory or a Hub repo id; bfloat16 weights by default
 
 doc = model.score(open("article.txt", encoding="utf-8").read(), title="Optional title")
@@ -193,40 +205,82 @@ python source1.py --model . --input docs.jsonl --text-field text --output scores
 python source1.py --model . --input page.txt --device cpu --precision fp32 --dtype fp32
 ```
 
-- **Output.** One flat dict per document: the 13 fields, `overall`, `keep`, `drop_reasons`, `parts` (chunks), the
-  length in tokens, whether a chunk was cut, and per-chunk results (`chunks`) for split documents.
-- **Precision.** `precision="bf16"` (default, `model.safetensors`, 0.6 GB) or `"fp32"` (`model.fp32.safetensors`,
-  1.2 GB, as trained; drop the `--exclude` above to get it, or load by Hub repo id, which downloads only the file you
-  ask for). It computes in bfloat16 on GPUs with native bfloat16 (NVIDIA Ampere or newer), as in the evaluation, and
-  in float32 elsewhere; float16 is refused. In bfloat16 scores move slightly with batch composition (up to about 0.04
-  on `overall`); use `dtype="fp32"` or `batch_tokens=1` to avoid it.
-- **Options.** `drop_line` (`"calibrated"` default, `"default"` for the rubric's hard filters, or an expression such
-  as `"toxicity >= 4 or spam_seo >= 3 or boilerplate >= 4.5"`), `apply_offsets`, `max_chunks`, `revision`;
-  `python source1.py --help` lists every command-line flag (JSONL, JSON or plain-text input).
-- **Examples and speed.** `examples/` holds six documents and their expected CPU output (a GPU can differ by a few
-  hundredths). Source-1 scores about 30 chunks (50,000 tokens) per second on one RTX 3090 in bfloat16.
-
-## What it returns
-
-| field | kind | scale or values |
+- **Output.** One flat dict per document with the 13 fields, `overall`, `keep` and `drop_reasons`. It also gives the
+  length in tokens, the number of chunks (`parts`), whether a chunk was cut, and per-chunk results (`chunks`).
+- **Precision.** By default it loads `model.safetensors` (bfloat16, 0.6 GB). For the full float32 weights (1.2 GB),
+  remove `--exclude` from the download and pass `precision="fp32"`. It computes in bfloat16 on NVIDIA Ampere or newer
+  GPUs and in float32 elsewhere. float16 is not supported. In bfloat16, scores can shift by up to about 0.04
+  depending on which texts share a batch; pass `dtype="fp32"` or `batch_tokens=1` to avoid that.
+- **Options.** `drop_line` sets your own drop rule, such as `"toxicity >= 4 or spam_seo >= 3 or boilerplate >= 4.5"`.
+  `max_chunks` scores only some chunks of very long documents, for speed. `revision` pins a Hub version. The command
+  line reads JSONL, JSON or plain text, and `python source1.py --help` lists every flag.
+- **Examples and speed.** `examples/` holds six documents and their expected CPU output. A GPU can differ by a few
+  hundredths. Source-1 scores about 30 chunks (50,000 tokens) per second on one RTX 3090 in bfloat16.
+
+**Files**
+
+| file | contents |
+|---|---|
+| `model.safetensors` | the fine-tuned mmBERT-base backbone in bfloat16 (default) |
+| `model.fp32.safetensors` | the same backbone in float32 (load with `precision="fp32"`) |
+| `config.json` | backbone configuration (ModernBERT) |
+| `heads.safetensors` | the 13 scoring heads |
+| `source1.json` | rubric, head layout, pooling, maximum length and text normalization |
+| `calibration.json` | the drop line and the quality-score offsets, with how they were chosen |
+| `tokenizer.json`, `tokenizer_config.json` | mmBERT's tokenizer (same vocabulary and merges, re-saved) |
+| `source1.py` | standalone loader, Python API and command line |
+| `requirements.txt` | `torch`, `transformers`, `safetensors`, `tokenizers` |
+| `examples/` | six sample inputs (`sample.jsonl`) and their expected command-line output (`expected_output.jsonl`) |
+| `EVALUATION.md` | the full evaluation, the rubric anchors and the training details |
+| `images/` | the benchmark chart above |
+| `LICENSE`, `NOTICE`, `AUTHORS` | license text, third-party notices and credits, authors |
+| `CREDITS_BOOKS.tsv` | per-work credits for the training books that are not public domain (part of `NOTICE`) |
+
+</details>
+
+## What you get
+
+| field | kind | what it measures |
 |---|---|---|
-| `format` | label | tutorial, reference, news, forum_qa, academic, fiction, code_file, product_page, blog_opinion, other |
-| `topic` | label | science, technology, programming, math, health, finance, history, politics_law, society, philosophy_religion, arts_entertainment, literature, sports, lifestyle, other |
-| `content_type` | label | plain_text, text_with_code, code_only, math_heavy |
-| `educational_value` | quality, 0-5, higher is better | Does it teach something useful? |
-| `reasoning_depth` | quality, 0-5 | Does it explain why and walk through steps, or just state facts? |
-| `writing_quality` | quality, 0-5 | Is it clear, well organized and coherent? (for code: naming, structure, comments) |
-| `information_density` | quality, 0-5 | How much real content per word, versus padding and filler? |
-| `reliability` | quality, 0-5 | Does it look careful and trustworthy, or sloppy and made up? (fiction is judged on care and consistency) |
-| `spam_seo` | red flag, 0-5, higher is worse | Text written to rank or sell rather than to inform: ads and pages that mainly promote score 3; keyword stuffing, clickbait and thin affiliate or doorway pages 4; auto-generated SEO text and scams 5 |
-| `boilerplate` | red flag, 0-5 | Templates, auto-generated pages, menus, link lists, cookie banners and other non-content text |
-| `toxicity` | red flag, 0-5 | Hate, harassment, explicit content (the text's own toxicity, not its subject) |
-| `code_quality` | gated, 0-5 or null | Is the code readable, correct-looking and documented, and written by a person rather than generated? Applies when `content_type` is text_with_code or code_only, or `format` is code_file |
-| `math_quality` | gated, 0-5 or null | Is the notation correct, are the steps shown, and do the solutions follow logically? Applies when `content_type` is math_heavy, or `topic` is math |
-
-Each 0-5 field is the expected level of a six-way head, a continuous number such as 2.73. Gated scores are null when
-the predicted labels say they do not apply. The anchors for every level are in `source1.json` and in
-[EVALUATION.md](EVALUATION.md#appendix-a-rubric-anchors), with the teacher's extra rules.
+| `format` | label | kind of text (10 types) |
+| `topic` | label | subject (15 topics) |
+| `content_type` | label | plain text, code or math |
+| `educational_value` | quality, 0-5 | teaches something useful |
+| `reasoning_depth` | quality, 0-5 | explains why, step by step |
+| `writing_quality` | quality, 0-5 | clear and well organized |
+| `information_density` | quality, 0-5 | real content, not filler |
+| `reliability` | quality, 0-5 | careful and trustworthy |
+| `spam_seo` | red flag, 0-5 | ads, SEO spam and scams |
+| `boilerplate` | red flag, 0-5 | menus, templates, link lists |
+| `toxicity` | red flag, 0-5 | hate, harassment, explicit content |
+| `code_quality` | code only, 0-5 | quality of the code |
+| `math_quality` | math only, 0-5 | quality of the math |
+
+Source-1 returns these 13 fields. Higher is better for quality scores and worse for red flags; the code and math
+scores are `null` when they do not apply. You also get:
+
+- `overall`: one 0-5 score, the quality scores minus penalties for red flags.
+- `keep`: false (drop) if `toxicity >= 4` or `spam_seo >= 3.5` or `boilerplate >= 4.5`.
+
+Long documents are split into chunks. Each chunk is scored, and the results are combined into one.
+
+<details>
+<summary>All fields in detail, the scoring formula, the drop line and long documents</summary>
+
+**Label values.**
+
+- `format`: tutorial, reference, news, forum_qa, academic, fiction, code_file, product_page, blog_opinion, other.
+- `topic`: science, technology, programming, math, health, finance, history, politics_law, society,
+  philosophy_religion, arts_entertainment, literature, sports, lifestyle, other.
+- `content_type`: plain_text, text_with_code, code_only, math_heavy.
+
+**Scores.** Each 0-5 score is a decimal such as 2.73: the model estimates how likely each level from 0 to 5 is, and
+the score is the average level weighted by those chances. `code_quality` applies when `content_type` is
+text_with_code or code_only, or `format` is code_file. `math_quality` applies when `content_type` is math_heavy or
+`topic` is math. Otherwise they are `null`. What each level means is in `source1.json` and
+[EVALUATION.md Appendix A](EVALUATION.md#appendix-a-rubric-anchors), with the extra rules the teacher was given.
+
+**Formula.**
 
 ```
 quality = 0.30*educational_value + 0.20*reasoning_depth + 0.15*writing_quality
@@ -239,34 +293,63 @@ overall = clip(quality - penalty, 0, 5)
 drop if  toxicity >= 4  or  spam_seo >= 3.5  or  boilerplate >= 4.5        (keep = false)
 ```
 
-The drop line in `calibration.json` is slightly stricter on spam than the rubric's own hard filters
-(`spam_seo >= 4`); both keep ads and promotional pages, which the rubric puts at `spam_seo` 3. It was picked on the
-teacher's validation labels by a fixed rule; stricter lines and their trade-offs are in
-[EVALUATION.md](EVALUATION.md#the-drop-line). You do not have to use `keep`: ranking on `overall`, or your own rules on
-single fields, may serve you better.
+**The drop line.** The line in `calibration.json` is a bit stricter on spam than the rubric's own rule
+(`spam_seo >= 4`). Both keep ads and promotional pages, which the rubric scores `spam_seo` 3. The line was chosen by a
+fixed rule on the teacher's labels. Stricter lines and what they cost are in
+[EVALUATION.md](EVALUATION.md#the-drop-line). You do not have to use `keep`: ranking by `overall`, or your own rules
+on single fields, may work better for you.
+
+**Long documents.** A document longer than about 7,800 tokens is split into balanced chunks at natural breaks. Each
+chunk is scored with a one-line header, as in training (source type, title if given, "Part i of n"). The document gets
+the label that covers the most tokens and the token-weighted average of each score. `toxicity` takes the maximum, and
+the code and math scores average only the chunks where they apply. `overall` and `keep` are then recomputed.
+
+</details>
 
-A document longer than about 7,800 tokens is split into balanced chunks at natural boundaries. Each chunk is scored
-after a one-line header, as in training (source type, title if given, "Part i of n"). The document then takes the
-label that covers the most tokens and the token-weighted mean of each score (the maximum for `toxicity`; gated scores
-over the chunks where they apply), and `overall` and `keep` are recomputed from those.
+## Good for / Not for
 
-## Intended use
+Good for:
+- Filtering, ranking, weighting and mixing pretraining text in its 53 languages, using any of its fields.
+- Checking a corpus: how much of it is spam, boilerplate, code, math or fiction.
+- Research on data quality and on teaching small models to copy LLM judgments.
 
-- Filtering, ranking and weighting text for language-model pretraining in the 53 training languages.
-- Building data mixtures from the labels and the individual scores, not just one number.
-- Auditing a corpus: how much of it is boilerplate, spam, code, math, fiction, and so on.
-- Research on data quality and on distilling LLM judgments into small encoders.
+Not for:
+- Judging people, job applications or student work.
+- Fact-checking or content moderation: `reliability` judges care, not truth, and `toxicity` is a rough data filter.
+- Deciding whether text is licensed or legal to use.
+- Other languages, non-text input, or generating text.
 
-Out of scope:
+## Limitations
 
-- Judging people, applications or student work. The scores describe text as training data.
-- Fact checking: `reliability` judges care and plausibility; it does not verify claims.
-- Content moderation or safety decisions: `toxicity` is a coarse flag for data filtering.
-- Deciding whether text is licensed, copyrighted or legal to use.
-- Languages outside the 53 listed (never trained or tested on), non-text inputs, and generating text.
+- **Home ground.** The grader used Source-1's own rubric. Models from the grader's family also helped shape that
+  rubric, pick the teacher and tune the drop rule. The test texts come from the same kinds of sources as the training
+  data. Whether filtering with Source-1 trains better models is untested.
+- **Two of the three test sets also helped pick the teacher**, by how well it agreed with the grader on them, so they
+  may flatter Source-1. The main test set was built after that and was not used to pick the model.
+- **propella-1 has no single score.** We combine four of its ratings. If we also count its red-flag ratings,
+  Source-1's lead shrinks by about half but remains ([details](EVALUATION.md#how-propella-1-is-read)).
+- **Scores run a bit high.** On English text it rates educational value, reasoning, writing and reliability about
+  0.3 points above the grader, as its teacher does.
+- **Its keep/drop rule catches about 2 of every 3 texts the grader drops.** A stricter rule catches more but also
+  drops more good text.
+- **Weaker outside web text** (books, chats, code, synthetic text). The code and math scores are the least reliable.
+- **No reproduction kit.** The test data, the grader's labels and the metrics script are not included.
+
+More in [EVALUATION.md](EVALUATION.md#limitations).
 
 ## Training
 
+- Fine-tuned from [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base), with 13 small scoring heads added.
+- About 173,000 text chunks in 53 languages, each labeled by an open-weight 27B LLM teacher (no human labels).
+- License and safety filters were applied: see [NOTICE](NOTICE) and [EVALUATION.md](EVALUATION.md#appendix-b-training-in-detail).
+- **Development note.** AI assistants helped write the rubric and the code. Grades from proprietary LLMs, including
+  the grader's model family, helped choose the rubric, the teacher and its setup, and the drop rule. LLM reviews also
+  helped shape the license filters. None of these grades or reviews was ever used as a label or training example
+  ([full account](EVALUATION.md#independence-from-development)).
+
+<details>
+<summary>Training details</summary>
+
 - **Base model.** [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) (ModernBERT architecture, 8,192-token
   context), all weights fine-tuned, with 13 linear heads on the mean-pooled hidden states: 307M parameters in total.
 - **Data.** 172,895 chunks (350M tokens, 151,281 documents) in 53 languages, 38.9% English: filtered and unfiltered
@@ -277,88 +360,22 @@ Out of scope:
 - **Recipe.** 2 epochs (5,320 steps of 131,072 tokens), AdamW, learning rate 5e-5 (heads 10x), bfloat16 mixed
   precision, about 1.9 hours on one H100 NVL. The learning rate and language mix came from a short sweep read on
   validation agreement with the teacher; the checkpoint is the final step, which also had the best validation score.
-- **Development note.** AI assistants helped draft the rubric and write the code. Grades from proprietary LLM graders,
-  among them models of the evaluation grader's family, were used to choose between rubric revisions, to vet the
-  teacher and choose its prompt setup (on the samples that became the exam sets), and to set the candidate drop lines
-  and the 0.95 floor of the drop-line rule; LLM reviews also informed the license filtering. None of these grades or
-  reviews was used as a label or training example.
 
 Data sources, filtering, recipe and development details:
 [EVALUATION.md](EVALUATION.md#appendix-b-training-in-detail) and
 [Independence from development](EVALUATION.md#independence-from-development).
 
-## Evaluation
-
-Source-1 was compared with 16 public quality scorers on three test sets, each graded by an independent proprietary
-LLM grader applying Source-1's rubric; the grader's labels were never trained on. Each cell is the rank agreement
-(Spearman) between each model's main score and the grader's overall score, on the chunks that model scored.
+</details>
 
-| test set | chunks | Source-1 | propella-1 4B | FineWeb-Edu classifier |
-|---|---|---|---|---|
-| held-out set, 53 languages (main result) | 495 | **0.900** | 0.756 | 0.529 (159 English chunks; Source-1 0.864) |
-| English exam | 412 to 413 | **0.921** | 0.820 | 0.453 |
-| 12-language exam (web text) | 350 to 352 | **0.895** | 0.637 | English only |
-
-The open-weight 27B LLM teacher scores 0.912, 0.912 and 0.880 on the same sets. All 39 public-scorer comparisons
-favor Source-1 with 95% intervals clear of zero. The held-out set is the main result because, unlike the exams, it
-played no part in choosing the teacher. Scoring each held-out chunk in full rather than its first 512 tokens raises
-agreement from 0.85 to 0.90 ([details](EVALUATION.md#reading-the-whole-chunk)).
-
-propella-1 has no single score: we read it through a composite of four of its quality ratings. Counting its own
-commercial-bias, content-ratio, integrity and safety ratings as well narrows its held-out gap from 0.14 to 0.06-0.08,
-depending on the weighting and languages, still in Source-1's favor
-([details](EVALUATION.md#how-propella-1-is-read)). The public scorers also read the text without Source-1's one-line
-header ([details](EVALUATION.md#the-one-line-header)).
-
-Every scorer, interval, drop-line result and caveat is in [EVALUATION.md](EVALUATION.md).
+## License and credits
 
-## Limitations
+- Source-1: [Apache License 2.0](LICENSE). Copyright 2026 The Source-1 Authors (see `AUTHORS`).
+- Base model: [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) by the mmBERT authors, MIT License.
+- Training data credits and third-party notices: [NOTICE](NOTICE) and `CREDITS_BOOKS.tsv`.
 
-- **Home-ground benchmark.** The grader applies Source-1's own rubric, and grades from its model family steered rubric
-  revisions, the choice of teacher and the drop-line design. The public scorers were built for their own definitions
-  of quality. Whether filtering with Source-1 trains better language models has not been tested.
-- **The exams helped choose the teacher.** The two exam sets are the samples on which the teacher and its prompt setup
-  were chosen, against the exam grader's labels, so they are not independent of the grader.
-- **It rates some qualities higher than the grader.** On the English exam's random sample, `educational_value` is 0.28
-  levels above the grader on average, and `reasoning_depth`, `writing_quality` and `reliability` 0.26 to 0.39 (the
-  teacher: 0.31, and 0.26 to 0.42).
-- **It misses a third of the grader's drops at the shipped line.** It catches 43 of 64 (0.67) on the held-out set and
-  the teacher 46; 17 of Source-1's 21 misses are also missed by the teacher. A stricter `drop_line` catches more.
-- **Weaker on some kinds of text.** Held-out rank is 0.68 for books and 0.73 for conversations, code and synthetic text,
-  against about 0.90 for web text; `code_quality` and `math_quality` are the least reliable fields.
-- **Not a fact checker or a moderation tool.** `reliability` is a surface judgment; `toxicity` was trained on data where
-  toxic text is rare.
-- **One chunk at a time, and less data for some languages.** Nothing outside a chunk of up to 8,192 tokens is visible to
-  it; the 16 languages with the fewest training chunks have 1,249 to 1,470 each.
-- **License screening has limits.** Notices worded in ways the patterns miss, and opt-outs outside the text (such as
-  robots.txt), were not caught. If you find such a document, tell us (see below).
-- **No reproduction kit.** The evaluation chunks, the grader's labels, per-chunk scores, the metrics script and the
-  teacher's prompt are not included, so the numbers cannot be recomputed from this repository.
-
-More: [Limitations in detail](EVALUATION.md#limitations-in-detail).
-
-## Files
+<details>
+<summary>License details</summary>
 
-| file | contents |
-|---|---|
-| `model.safetensors` | the fine-tuned mmBERT-base backbone in bfloat16 (default) |
-| `model.fp32.safetensors` | the same backbone in float32 (load with `precision="fp32"`) |
-| `config.json` | backbone configuration (ModernBERT) |
-| `heads.safetensors` | the 13 scoring heads |
-| `source1.json` | rubric, head layout, pooling, maximum length and text normalization |
-| `calibration.json` | the drop line and the quality-score offsets, with how they were chosen |
-| `tokenizer.json`, `tokenizer_config.json` | mmBERT's tokenizer (same vocabulary and merges, re-saved) |
-| `source1.py` | standalone loader, Python API and command line |
-| `requirements.txt` | `torch`, `transformers`, `safetensors`, `tokenizers` |
-| `examples/` | six sample inputs (`sample.jsonl`) and their expected command-line output (`expected_output.jsonl`) |
-| `EVALUATION.md` | the full evaluation, the rubric anchors and the training details |
-| `images/` | the benchmark chart above |
-| `LICENSE`, `NOTICE`, `AUTHORS` | license text, third-party notices and credits, authors |
-| `CREDITS_BOOKS.tsv` | per-work credits for the training books that are not public domain (part of `NOTICE`) |
-
-## License and credits
-
-Source-1 is released under the [Apache License 2.0](LICENSE). Copyright 2026 The Source-1 Authors (see `AUTHORS`).
 [`NOTICE`](NOTICE) holds the full third-party notices and data credits. In short:
 
 - **Base model.** Fine-tuned from [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) by the mmBERT authors at
@@ -377,14 +394,18 @@ Source-1 is released under the [Apache License 2.0](LICENSE). Copyright 2026 The
   text it was trained on. Copyleft code was removed anyway.
 - Upstream license metadata can be wrong. If you find a source that should not be here, please tell us (below).
 
-## Contact and takedown
+</details>
 
-Questions, corrections and removal requests: the [Community tab](https://huggingface.co/msmth/Source-1/discussions). If you believe your content was used to train Source-1 and
-you want it excluded from future versions, tell us the URL or dataset and we will remove it from the training data of
-the next release.
+## Contact
+
+Questions, corrections and removal requests: the [Community tab](https://huggingface.co/msmth/Source-1/discussions).
+Send the URL or dataset of your content and we will remove it from the next release's training data.
 
 ## Citation
 
+<details>
+<summary>BibTeX</summary>
+
 ```bibtex
 @misc{source1_2026,
   title        = {Source-1: a multilingual 13-field scorer for pretraining data},
@@ -407,3 +428,5 @@ Please also cite mmBERT:
   url           = {https://arxiv.org/abs/2509.06888}
 }
 ```
+
+</details>

source1.py

@@ -51,7 +51,7 @@ How a document is scored (the same steps the model was trained and evaluated wit
    Every training input had a Source line, almost always "dataset record", so that is the default; code files had
    "<language> source file" (pass ``code_language="Python"``). Title is added when given, Part when the document has
    more than one chunk. ``url`` is accepted but not shown to the model unless ``show_url=True``: no training input had
-   one. On 1,261 held-out benchmark chunks (495 graded blind-set chunks from the test split and 766 exam chunks),
+   one. On 1,261 held-out benchmark chunks (495 graded held-out chunks from the test split from the test split and 766 exam chunks),
    dropping the whole header moved overall by 0.03 on average (at most 0.655), dropping a title by 0.05 on the chunks
    that had one; adding a URL moved it by up to 0.7 and did not improve its rank agreement with the independent
    graders of the model card's evaluation.