Upload README.md with huggingface_hub

README.md

@@ -1,60 +1,82 @@
 ---
-title: Eval Suite Visualization
+title: ellamind base-eval
 emoji: 📊
 colorFrom: blue
 colorTo: indigo
 sdk: static
 pinned: false
+hf_oauth: true
+hf_oauth_scopes:
+  - read-repos
 ---
 
-# Eval Suite Visualization
+# ellamind base-eval
 
-A static web app for visualizing LLM evaluation scores. Data is loaded directly from a HuggingFace dataset ([ellamind/eval-scores](https://huggingface.co/datasets/ellamind/eval-scores)) using DuckDB-WASM — no preprocessing or backend required.
+Interactive visualization for LLM evaluation scores during pre-training. Data is loaded from HuggingFace datasets via DuckDB-WASM — no backend required.
+
+- **Default dataset**: [ellamind/eval-scores-ref](https://huggingface.co/datasets/ellamind/eval-scores-ref)
+- **GitHub**: [ellamind/base-eval](https://github.com/ellamind/base-eval)
 
 ## Features
 
-- **Hierarchical task selection**: eval suite → task group → individual benchmark, with aggregate views
-- **Multiple metrics**: `acc`, `acc_norm`, `bits_per_byte`, `exact_match`, `pass@1`, etc.
-- **Model comparison**: toggle models on/off; separate checkpoint runs from baselines
-- **Auto chart type**: line charts for training runs (tokens trained on x-axis), bar charts for single-point comparisons
-- **Multi-panel layout**: add multiple independent panels side by side
-- **Smoothing**: configurable moving average for line charts
-- **Export**: download charts as PNG or SVG
+- **Hierarchical task selection** — eval suite → task group → individual benchmark, with aggregate views
+- **Multiple metrics** — `acc`, `acc_norm`, `bits_per_byte`, `exact_match`, `pass@1`, etc.
+- **Model comparison** — toggle models on/off; separate checkpoint runs from baselines
+- **Auto chart type** — line charts for training runs (tokens trained on x-axis), bar charts for single-point comparisons
+- **Multi-panel layout** — add multiple independent panels
+- **Merge datasets** — append rows from additional HF datasets (including private ones via OAuth)
+- **Smoothing** — configurable moving average for line charts
+- **Benchmark goodness metrics** — per-task quality indicators below line charts
+- **Export** — download charts as PNG or SVG
 
-## Quick Start
+## Merge Datasets
 
-Serve the app with any static file server:
+You can merge additional HF datasets into the visualization at runtime. Enter a dataset path (e.g. `org/dataset-name` or `org/dataset-name/custom.parquet`) and click **Merge Dataset**. The additional data is row-appended to the base dataset.
 
-```bash
-python3 -m http.server 8080
-```
+For **private datasets**, sign in with your HuggingFace account using the OAuth button. The access token is used automatically when fetching.
 
-Then open `http://localhost:8080`. The app fetches the parquet data directly from HuggingFace on load.
+## Benchmark Goodness Metrics
 
-## Project Structure
+Line charts display quality indicators below the plot, inspired by the [FineTasks](https://huggingface.co/spaces/HuggingFaceFW/blogpost-fine-tasks) methodology. Metrics are computed client-side across three stages: **Overall**, **Early** (first half of training), and **Late** (second half).
 
-```
-index.html    # Single-file web app (HTML + CSS + JS)
-config.yaml   # Model color overrides
-README.md     # HF Spaces metadata + docs
-```
+| Metric | What it measures | Green | Yellow | Red |
+|---|---|---|---|---|
+| **Monotonicity** | Spearman correlation between steps and score | ≥ 0.7 | 0.4–0.7 | < 0.4 |
+| **Signal Strength** | Relative improvement over initial performance | ≥ 0.10 | 0.03–0.10 | < 0.03 |
+| **Noise** | MAD of consecutive score diffs (robust to data-mix jumps) | — | — | — |
+| **Ordering** | Kendall's Tau of model rankings between steps | ≥ 0.6 | 0.3–0.6 | < 0.3 |
+| **Discrimination** | Std of scores across models at last checkpoint | ≥ 0.03 | 0.01–0.03 | < 0.01 |
 
 ## Configuration
 
-Model colors can be customized in `config.yaml`:
+Model colors in `config.yaml`:
 
 ```yaml
 model_colors:
-  "D01": "#4361ee"
-  "Qwen3 1.7B": "#6F53D1"
+  "Qwen3 1.7B": "#9575CD"
+  "Gemma 3 4B": "#00B0FF"
 ```
 
-Exact matches are checked first, then prefix matches. Models without a configured color get assigned one from a default palette.
+## Local Development
 
-## Deployment
+```bash
+python3 -m http.server 8080
+```
+
+OAuth sign-in is only available when deployed as an HF Space. Locally, it is hidden.
 
-This app is deployed as a [Static HTML Space](https://huggingface.co/docs/hub/spaces-sdks-static) on Hugging Face. To deploy:
+## Deployment
 
 ```bash
-huggingface-cli upload ellamind/eval-suite-visualization . . --repo-type space
+pixi run -- hf upload ellamind/base-eval index.html index.html --repo-type space
+pixi run -- hf upload ellamind/base-eval config.yaml config.yaml --repo-type space
+pixi run -- hf upload ellamind/base-eval README.md README.md --repo-type space
+```
+
+## Project Structure
+
+```
+index.html    # Single-file web app (HTML + CSS + JS)
+config.yaml   # Model color overrides
+README.md     # HF Spaces metadata + docs
 ```