Spaces:
Paused
Paused
init nba-evo-s21 (darwinian_weights) — S20/S21/S22 SOTA genotype
Browse files- Dockerfile.browser +64 -0
- README.md +23 -7
- app.py +0 -0
- data/calibration-map.json +45 -0
- data/historical/games-2018-19.json +0 -0
- data/historical/games-2019-20.json +0 -0
- data/historical/games-2020-21.json +0 -0
- data/historical/games-2021-22.json +0 -0
- data/historical/games-2022-23.json +0 -0
- data/historical/games-2023-24.json +0 -0
- data/historical/games-2024-25.json +0 -0
- data/historical/games-2025-26.json +0 -0
- evolution/__init__.py +0 -0
- evolution/genetic_loop_v3.py +2002 -0
- evolution/run_logger.py +425 -0
- evolution/sota_s21.py +108 -0
- experiment_runner.py +1073 -0
- features/__init__.py +0 -0
- features/engine.py +0 -0
- models/__init__.py +35 -0
- models/neural_models.py +1598 -0
- requirements.txt +19 -0
Dockerfile.browser
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dockerfile.browser -- Browser-enabled HF Space image for web scraping
|
| 2 |
+
# ======================================================================
|
| 3 |
+
#
|
| 4 |
+
# NOT ACTIVE YET -- this is a template for when we need browser-based
|
| 5 |
+
# scraping on HF Spaces (e.g., scraping odds pages with JS rendering).
|
| 6 |
+
#
|
| 7 |
+
# Current HF Spaces use the default Python runtime without browser deps.
|
| 8 |
+
# To activate: rename to Dockerfile and push to the target Space.
|
| 9 |
+
#
|
| 10 |
+
# Requirements:
|
| 11 |
+
# - HF Space must be configured as "Docker" SDK (not Gradio SDK)
|
| 12 |
+
# - The Space will be larger (~2GB) due to Chromium
|
| 13 |
+
# - CPU-only is fine for scraping (no GPU needed)
|
| 14 |
+
#
|
| 15 |
+
# Size estimate: ~2.5GB image (Playwright + Chromium + Python deps)
|
| 16 |
+
|
| 17 |
+
FROM python:3.11-slim-bookworm
|
| 18 |
+
|
| 19 |
+
# Install system deps for Playwright/Chromium
|
| 20 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 21 |
+
wget \
|
| 22 |
+
ca-certificates \
|
| 23 |
+
fonts-liberation \
|
| 24 |
+
libasound2 \
|
| 25 |
+
libatk-bridge2.0-0 \
|
| 26 |
+
libatk1.0-0 \
|
| 27 |
+
libcups2 \
|
| 28 |
+
libdbus-1-3 \
|
| 29 |
+
libdrm2 \
|
| 30 |
+
libgbm1 \
|
| 31 |
+
libgtk-3-0 \
|
| 32 |
+
libnspr4 \
|
| 33 |
+
libnss3 \
|
| 34 |
+
libx11-xcb1 \
|
| 35 |
+
libxcomposite1 \
|
| 36 |
+
libxdamage1 \
|
| 37 |
+
libxrandr2 \
|
| 38 |
+
xdg-utils \
|
| 39 |
+
&& rm -rf /var/lib/apt/lists/*
|
| 40 |
+
|
| 41 |
+
# Create non-root user (HF Spaces requirement)
|
| 42 |
+
RUN useradd -m -u 1000 user
|
| 43 |
+
WORKDIR /home/user/app
|
| 44 |
+
|
| 45 |
+
# Install Python deps
|
| 46 |
+
COPY requirements.txt .
|
| 47 |
+
RUN pip install --no-cache-dir -r requirements.txt
|
| 48 |
+
|
| 49 |
+
# Install Playwright and Chromium browser
|
| 50 |
+
RUN pip install --no-cache-dir crawl4ai>=0.4 playwright
|
| 51 |
+
RUN playwright install chromium
|
| 52 |
+
RUN playwright install-deps chromium
|
| 53 |
+
|
| 54 |
+
# Copy application code
|
| 55 |
+
COPY . .
|
| 56 |
+
|
| 57 |
+
# Fix permissions
|
| 58 |
+
RUN chown -R user:user /home/user/app
|
| 59 |
+
|
| 60 |
+
USER user
|
| 61 |
+
|
| 62 |
+
EXPOSE 7860
|
| 63 |
+
|
| 64 |
+
CMD ["python", "app.py"]
|
README.md
CHANGED
|
@@ -1,12 +1,28 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
app_file: app.py
|
| 9 |
-
pinned:
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: NBA Evo S21 — Darwinian Weights
|
| 3 |
+
emoji: 🧬
|
| 4 |
+
colorFrom: green
|
| 5 |
+
colorTo: purple
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 5.12.0
|
| 8 |
app_file: app.py
|
| 9 |
+
pinned: true
|
| 10 |
+
short_description: "S21 — Darwinian PnL-weighted ensemble (atlas-gic)"
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# NBA Evo S21 — Darwinian Weights (atlas-gic inspired)
|
| 14 |
+
|
| 15 |
+
Ensemble weights for base models are driven by rolling 30-day PnL rather
|
| 16 |
+
than validation log-loss. Weights are recomputed every 10 generations with
|
| 17 |
+
exponential decay alpha=0.9. Inspiration: atlas-gic (research_cycle7_sota_gap.md).
|
| 18 |
+
|
| 19 |
+
# NOMOS NBA Quant AI — Continuous Training
|
| 20 |
+
|
| 21 |
+
Always-running agentic loop for NBA quantitative prediction models.
|
| 22 |
+
|
| 23 |
+
- **5 tree-based models** evolved via NSGA-II genetic algorithm (CPU-optimized)
|
| 24 |
+
- **8 seasons** of NBA data (9,551+ games, 2018-2026)
|
| 25 |
+
- **Up to 200 features** (island-specific: 55-80) from v3.0-37cat engine with MOVDA
|
| 26 |
+
- **Walk-forward backtesting** with Kelly criterion sizing
|
| 27 |
+
- **Feature engine**: v3.0 + Cat36 EWMA + Cat37 MOVDA (deployed 2026-03-25)
|
| 28 |
+
- **MAX_FEATURES=200** hard cap, mutation capped at 0.15, xgboost_brier fixed
|
app.py
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/calibration-map.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_meta": {
|
| 3 |
+
"version": "2.0",
|
| 4 |
+
"created": "2026-04-10",
|
| 5 |
+
"generated_at": "2026-04-10T19:35:17.183416+00:00",
|
| 6 |
+
"source": "scripts/calibration_fit.py (Pool Adjacent Violators + empirical blend)",
|
| 7 |
+
"model_version": "ensemble v1 / real_predictions_loader",
|
| 8 |
+
"notes": "Rebuilt from real matched predictions (prospective, no look-ahead). Previous 31-game hand-tuned map over-corrected bin 6 0.65->0.35. This fit uses 70% PAV curve + 30% empirical bin rate to borrow strength from neighbouring bins in low-count regions.",
|
| 9 |
+
"n_games_used": 104,
|
| 10 |
+
"date_range": "2026-03-16 to 2026-04-05",
|
| 11 |
+
"brier_before": 0.24246,
|
| 12 |
+
"brier_after": 0.23057,
|
| 13 |
+
"ece_before": 0.123,
|
| 14 |
+
"ece_after": 0.06217
|
| 15 |
+
},
|
| 16 |
+
"bin_edges": [
|
| 17 |
+
0.0,
|
| 18 |
+
0.2,
|
| 19 |
+
0.4,
|
| 20 |
+
0.6,
|
| 21 |
+
0.8,
|
| 22 |
+
1.0
|
| 23 |
+
],
|
| 24 |
+
"bin_counts": [
|
| 25 |
+
4,
|
| 26 |
+
20,
|
| 27 |
+
26,
|
| 28 |
+
44,
|
| 29 |
+
10
|
| 30 |
+
],
|
| 31 |
+
"raw_centers": [
|
| 32 |
+
0.1,
|
| 33 |
+
0.3,
|
| 34 |
+
0.5,
|
| 35 |
+
0.7,
|
| 36 |
+
0.9
|
| 37 |
+
],
|
| 38 |
+
"calibrated_centers": [
|
| 39 |
+
0.075,
|
| 40 |
+
0.4417,
|
| 41 |
+
0.5365,
|
| 42 |
+
0.5556,
|
| 43 |
+
0.97
|
| 44 |
+
]
|
| 45 |
+
}
|
data/historical/games-2018-19.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2019-20.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2020-21.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2021-22.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2022-23.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2023-24.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2024-25.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
data/historical/games-2025-26.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
evolution/__init__.py
ADDED
|
File without changes
|
evolution/genetic_loop_v3.py
ADDED
|
@@ -0,0 +1,2002 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
NBA Quant AI — REAL Genetic Evolution Loop v4
|
| 4 |
+
================================================
|
| 5 |
+
RUNS 24/7 on HF Space or Google Colab.
|
| 6 |
+
|
| 7 |
+
This is NOT a fake LLM wrapper. This is REAL ML:
|
| 8 |
+
- Population of 500 individuals across 5 islands (100 per island)
|
| 9 |
+
- 13 model types: tree-based + neural nets (LSTM, Transformer, TabNet, etc.)
|
| 10 |
+
- NSGA-II Pareto front ranking (multi-objective: Brier, ROI, Sharpe, Calibration)
|
| 11 |
+
- Island migration every 10 generations for diversity
|
| 12 |
+
- Adaptive mutation: 0.15 -> 0.05 decay + stagnation boost
|
| 13 |
+
- Memory management: GC between evaluations for 16GB RAM
|
| 14 |
+
- Continuous cycles — saves after each generation
|
| 15 |
+
- Callbacks to VM after each cycle
|
| 16 |
+
- Population persistence (survives restarts)
|
| 17 |
+
|
| 18 |
+
Usage:
|
| 19 |
+
# On HF Space (24/7):
|
| 20 |
+
python evolution/genetic_loop_v3.py --continuous
|
| 21 |
+
|
| 22 |
+
# On Google Colab (manual):
|
| 23 |
+
!python genetic_loop_v3.py --generations 50
|
| 24 |
+
|
| 25 |
+
# Quick test:
|
| 26 |
+
python evolution/genetic_loop_v3.py --generations 5 --pop-size 50
|
| 27 |
+
"""
|
| 28 |
+
|
| 29 |
+
import os, sys, json, time, random, math, warnings, traceback, gc
|
| 30 |
+
import numpy as np
|
| 31 |
+
from pathlib import Path
|
| 32 |
+
from datetime import datetime, timezone, timedelta
|
| 33 |
+
from collections import defaultdict
|
| 34 |
+
from typing import Dict, List, Tuple, Optional
|
| 35 |
+
|
| 36 |
+
warnings.filterwarnings("ignore")
|
| 37 |
+
|
| 38 |
+
# All model types the GA can evolve
|
| 39 |
+
CPU_MODEL_TYPES = [
|
| 40 |
+
"xgboost", "xgboost_brier", "lightgbm", "catboost", "random_forest", "extra_trees",
|
| 41 |
+
]
|
| 42 |
+
GPU_MODEL_TYPES = CPU_MODEL_TYPES + ["tabicl", "tabpfn"]
|
| 43 |
+
ALL_MODEL_TYPES = GPU_MODEL_TYPES + [
|
| 44 |
+
"stacking", "mlp", "lstm", "transformer", "tabnet",
|
| 45 |
+
"ft_transformer", "deep_ensemble", "autogluon",
|
| 46 |
+
]
|
| 47 |
+
NEURAL_NET_TYPES = {"lstm", "transformer", "tabnet", "ft_transformer", "deep_ensemble", "mlp", "autogluon"}
|
| 48 |
+
ICL_MODEL_TYPES = {"tabicl", "tabpfn"} # In-context learning models (GPU, no hyperparams to tune)
|
| 49 |
+
|
| 50 |
+
# ── Run Logger (best-effort) ──
|
| 51 |
+
try:
|
| 52 |
+
from evolution.run_logger import RunLogger
|
| 53 |
+
_HAS_LOGGER = True
|
| 54 |
+
except ImportError:
|
| 55 |
+
try:
|
| 56 |
+
from run_logger import RunLogger
|
| 57 |
+
_HAS_LOGGER = True
|
| 58 |
+
except ImportError:
|
| 59 |
+
_HAS_LOGGER = False
|
| 60 |
+
|
| 61 |
+
# ─── Auto-load .env.local ───
|
| 62 |
+
_env_file = Path(__file__).resolve().parent.parent / ".env.local"
|
| 63 |
+
if not _env_file.exists():
|
| 64 |
+
_env_file = Path("/app/.env.local")
|
| 65 |
+
if _env_file.exists():
|
| 66 |
+
for _line in _env_file.read_text().splitlines():
|
| 67 |
+
_line = _line.strip()
|
| 68 |
+
if _line and not _line.startswith("#") and "=" in _line:
|
| 69 |
+
_line = _line.replace("export ", "")
|
| 70 |
+
_k, _, _v = _line.partition("=")
|
| 71 |
+
os.environ.setdefault(_k.strip(), _v.strip("'\""))
|
| 72 |
+
|
| 73 |
+
# ─── Paths ───
|
| 74 |
+
BASE_DIR = Path(__file__).resolve().parent.parent
|
| 75 |
+
DATA_DIR = BASE_DIR / "data"
|
| 76 |
+
HIST_DIR = DATA_DIR / "historical"
|
| 77 |
+
RESULTS_DIR = DATA_DIR / "results"
|
| 78 |
+
STATE_DIR = DATA_DIR / "evolution-state"
|
| 79 |
+
for d in [DATA_DIR, HIST_DIR, RESULTS_DIR, STATE_DIR]:
|
| 80 |
+
d.mkdir(parents=True, exist_ok=True)
|
| 81 |
+
|
| 82 |
+
VM_CALLBACK_URL = os.environ.get("VM_CALLBACK_URL", "http://34.136.180.66:8080")
|
| 83 |
+
ODDS_API_KEY = os.environ.get("ODDS_API_KEY", "")
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
# ═══════════════════════════════════════════════════════════
|
| 87 |
+
# SECTION 1: DATA LOADING
|
| 88 |
+
# ═══════════════════════════════════════════════════════════
|
| 89 |
+
|
| 90 |
+
TEAM_MAP = {
|
| 91 |
+
"Atlanta Hawks": "ATL", "Boston Celtics": "BOS", "Brooklyn Nets": "BKN",
|
| 92 |
+
"Charlotte Hornets": "CHA", "Chicago Bulls": "CHI", "Cleveland Cavaliers": "CLE",
|
| 93 |
+
"Dallas Mavericks": "DAL", "Denver Nuggets": "DEN", "Detroit Pistons": "DET",
|
| 94 |
+
"Golden State Warriors": "GSW", "Houston Rockets": "HOU", "Indiana Pacers": "IND",
|
| 95 |
+
"Los Angeles Clippers": "LAC", "Los Angeles Lakers": "LAL", "Memphis Grizzlies": "MEM",
|
| 96 |
+
"Miami Heat": "MIA", "Milwaukee Bucks": "MIL", "Minnesota Timberwolves": "MIN",
|
| 97 |
+
"New Orleans Pelicans": "NOP", "New York Knicks": "NYK", "Oklahoma City Thunder": "OKC",
|
| 98 |
+
"Orlando Magic": "ORL", "Philadelphia 76ers": "PHI", "Phoenix Suns": "PHX",
|
| 99 |
+
"Portland Trail Blazers": "POR", "Sacramento Kings": "SAC", "San Antonio Spurs": "SAS",
|
| 100 |
+
"Toronto Raptors": "TOR", "Utah Jazz": "UTA", "Washington Wizards": "WAS",
|
| 101 |
+
}
|
| 102 |
+
|
| 103 |
+
ARENA_COORDS = {
|
| 104 |
+
"ATL": (33.757, -84.396), "BOS": (42.366, -71.062), "BKN": (40.683, -73.976),
|
| 105 |
+
"CHA": (35.225, -80.839), "CHI": (41.881, -87.674), "CLE": (41.496, -81.688),
|
| 106 |
+
"DAL": (32.790, -96.810), "DEN": (39.749, -105.008), "DET": (42.341, -83.055),
|
| 107 |
+
"GSW": (37.768, -122.388), "HOU": (29.751, -95.362), "IND": (39.764, -86.156),
|
| 108 |
+
"LAC": (34.043, -118.267), "LAL": (34.043, -118.267), "MEM": (35.138, -90.051),
|
| 109 |
+
"MIA": (25.781, -80.187), "MIL": (43.045, -87.917), "MIN": (44.980, -93.276),
|
| 110 |
+
"NOP": (29.949, -90.082), "NYK": (40.751, -73.994), "OKC": (35.463, -97.515),
|
| 111 |
+
"ORL": (28.539, -81.384), "PHI": (39.901, -75.172), "PHX": (33.446, -112.071),
|
| 112 |
+
"POR": (45.532, -122.667), "SAC": (38.580, -121.500), "SAS": (29.427, -98.438),
|
| 113 |
+
"TOR": (43.643, -79.379), "UTA": (40.768, -111.901), "WAS": (38.898, -77.021),
|
| 114 |
+
}
|
| 115 |
+
|
| 116 |
+
ARENA_ALTITUDE = {
|
| 117 |
+
"DEN": 5280, "UTA": 4226, "PHX": 1086, "OKC": 1201, "SAS": 650,
|
| 118 |
+
"DAL": 430, "HOU": 43, "MEM": 337, "ATL": 1050, "CHA": 751,
|
| 119 |
+
"IND": 715, "CHI": 594, "MIL": 617, "MIN": 830, "DET": 600,
|
| 120 |
+
"CLE": 653, "BOS": 141, "NYK": 33, "BKN": 33, "PHI": 39,
|
| 121 |
+
"WAS": 25, "MIA": 6, "ORL": 82, "NOP": 7, "TOR": 250,
|
| 122 |
+
"POR": 50, "SAC": 30, "GSW": 12, "LAL": 305, "LAC": 305,
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
TIMEZONE_ET = {
|
| 126 |
+
"ATL": 0, "BOS": 0, "BKN": 0, "CHA": 0, "CHI": -1, "CLE": 0,
|
| 127 |
+
"DAL": -1, "DEN": -2, "DET": 0, "GSW": -3, "HOU": -1, "IND": 0,
|
| 128 |
+
"LAC": -3, "LAL": -3, "MEM": -1, "MIA": 0, "MIL": -1, "MIN": -1,
|
| 129 |
+
"NOP": -1, "NYK": 0, "OKC": -1, "ORL": 0, "PHI": 0, "PHX": -2,
|
| 130 |
+
"POR": -3, "SAC": -3, "SAS": -1, "TOR": 0, "UTA": -2, "WAS": 0,
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
WINDOWS = [3, 5, 7, 10, 15, 20]
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
def resolve(name):
|
| 137 |
+
if name in TEAM_MAP: return TEAM_MAP[name]
|
| 138 |
+
if len(name) == 3 and name.isupper(): return name
|
| 139 |
+
for full, abbr in TEAM_MAP.items():
|
| 140 |
+
if name in full: return abbr
|
| 141 |
+
return name[:3].upper() if name else None
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def haversine(lat1, lon1, lat2, lon2):
|
| 145 |
+
R = 3959
|
| 146 |
+
dlat, dlon = math.radians(lat2 - lat1), math.radians(lon2 - lon1)
|
| 147 |
+
a = math.sin(dlat/2)**2 + math.cos(math.radians(lat1)) * math.cos(math.radians(lat2)) * math.sin(dlon/2)**2
|
| 148 |
+
return R * 2 * math.asin(math.sqrt(a))
|
| 149 |
+
|
| 150 |
+
|
| 151 |
+
def pull_seasons():
|
| 152 |
+
"""Pull NBA game data from nba_api, cache locally."""
|
| 153 |
+
try:
|
| 154 |
+
from nba_api.stats.endpoints import leaguegamefinder
|
| 155 |
+
except ImportError:
|
| 156 |
+
print("[DATA] nba_api not installed, using cached data only")
|
| 157 |
+
return
|
| 158 |
+
|
| 159 |
+
existing = {f.stem.replace("games-", "") for f in HIST_DIR.glob("games-*.json")}
|
| 160 |
+
targets = ["2018-19", "2019-20", "2020-21", "2021-22", "2022-23", "2023-24", "2024-25", "2025-26"]
|
| 161 |
+
missing = [s for s in targets if s not in existing]
|
| 162 |
+
if not missing:
|
| 163 |
+
print(f"[DATA] All {len(targets)} seasons cached")
|
| 164 |
+
return
|
| 165 |
+
|
| 166 |
+
for season in missing:
|
| 167 |
+
print(f"[DATA] Pulling {season}...")
|
| 168 |
+
try:
|
| 169 |
+
time.sleep(3)
|
| 170 |
+
finder = leaguegamefinder.LeagueGameFinder(
|
| 171 |
+
season_nullable=season, league_id_nullable="00",
|
| 172 |
+
season_type_nullable="Regular Season", timeout=60
|
| 173 |
+
)
|
| 174 |
+
df = finder.get_data_frames()[0]
|
| 175 |
+
if df.empty:
|
| 176 |
+
continue
|
| 177 |
+
pairs = {}
|
| 178 |
+
for _, row in df.iterrows():
|
| 179 |
+
gid = row["GAME_ID"]
|
| 180 |
+
if gid not in pairs:
|
| 181 |
+
pairs[gid] = []
|
| 182 |
+
pairs[gid].append({
|
| 183 |
+
"team_name": row.get("TEAM_NAME", ""),
|
| 184 |
+
"matchup": row.get("MATCHUP", ""),
|
| 185 |
+
"pts": int(row["PTS"]) if row.get("PTS") is not None else None,
|
| 186 |
+
"game_date": row.get("GAME_DATE", ""),
|
| 187 |
+
})
|
| 188 |
+
games = []
|
| 189 |
+
for gid, teams in pairs.items():
|
| 190 |
+
if len(teams) != 2:
|
| 191 |
+
continue
|
| 192 |
+
home = next((t for t in teams if " vs. " in str(t.get("matchup", ""))), None)
|
| 193 |
+
away = next((t for t in teams if " @ " in str(t.get("matchup", ""))), None)
|
| 194 |
+
if not home or not away or home["pts"] is None:
|
| 195 |
+
continue
|
| 196 |
+
games.append({
|
| 197 |
+
"game_date": home["game_date"],
|
| 198 |
+
"home_team": home["team_name"], "away_team": away["team_name"],
|
| 199 |
+
"home": {"team_name": home["team_name"], "pts": home["pts"]},
|
| 200 |
+
"away": {"team_name": away["team_name"], "pts": away["pts"]},
|
| 201 |
+
})
|
| 202 |
+
if games:
|
| 203 |
+
(HIST_DIR / f"games-{season}.json").write_text(json.dumps(games))
|
| 204 |
+
print(f" {len(games)} games saved")
|
| 205 |
+
except Exception as e:
|
| 206 |
+
print(f" Error pulling {season}: {e}")
|
| 207 |
+
|
| 208 |
+
|
| 209 |
+
def load_all_games():
|
| 210 |
+
"""Load all cached game data."""
|
| 211 |
+
games = []
|
| 212 |
+
for f in sorted(HIST_DIR.glob("games-*.json")):
|
| 213 |
+
data = json.loads(f.read_text())
|
| 214 |
+
items = data if isinstance(data, list) else data.get("games", [])
|
| 215 |
+
games.extend(items)
|
| 216 |
+
games.sort(key=lambda g: g.get("game_date", g.get("date", "")))
|
| 217 |
+
return games
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
# ═══════════════════════════════════════════════════════════
|
| 221 |
+
# SECTION 2: FEATURE ENGINE
|
| 222 |
+
# ═══════════════════════════════════════════════════════════
|
| 223 |
+
|
| 224 |
+
FEATURE_ENGINE_VERSION = "genetic-loop-v3"
|
| 225 |
+
|
| 226 |
+
def build_features(games):
|
| 227 |
+
"""Build features from raw game data. Tries real NBAFeatureEngine first, falls back to inline."""
|
| 228 |
+
try:
|
| 229 |
+
from features.engine import NBAFeatureEngine
|
| 230 |
+
engine = NBAFeatureEngine(skip_placeholder=True)
|
| 231 |
+
X, y, feature_names = engine.build(games)
|
| 232 |
+
X = np.nan_to_num(np.array(X, dtype=np.float64), nan=0.0, posinf=1e6, neginf=-1e6)
|
| 233 |
+
y = np.array(y, dtype=np.int32)
|
| 234 |
+
print(f"[ENGINE] Real NBAFeatureEngine: {X.shape[1]} features, {len(y)} games")
|
| 235 |
+
return X, y, feature_names
|
| 236 |
+
except Exception as e:
|
| 237 |
+
print(f"[ENGINE] NBAFeatureEngine import failed ({e}), using inline fallback")
|
| 238 |
+
return _build_features_inline(games)
|
| 239 |
+
|
| 240 |
+
|
| 241 |
+
def _build_features_inline(games):
|
| 242 |
+
"""Fallback: Build 250+ inline features from raw game data. Returns X, y, feature_names."""
|
| 243 |
+
team_results = defaultdict(list)
|
| 244 |
+
team_last = {}
|
| 245 |
+
team_elo = defaultdict(lambda: 1500.0)
|
| 246 |
+
X, y = [], []
|
| 247 |
+
feature_names = []
|
| 248 |
+
first = True
|
| 249 |
+
|
| 250 |
+
for game in games:
|
| 251 |
+
hr, ar = game.get("home_team", ""), game.get("away_team", "")
|
| 252 |
+
if "home" in game and isinstance(game["home"], dict):
|
| 253 |
+
h, a = game["home"], game.get("away", {})
|
| 254 |
+
hs, as_ = h.get("pts"), a.get("pts")
|
| 255 |
+
if not hr: hr = h.get("team_name", "")
|
| 256 |
+
if not ar: ar = a.get("team_name", "")
|
| 257 |
+
else:
|
| 258 |
+
hs, as_ = game.get("home_score"), game.get("away_score")
|
| 259 |
+
if hs is None or as_ is None:
|
| 260 |
+
continue
|
| 261 |
+
hs, as_ = int(hs), int(as_)
|
| 262 |
+
home, away = resolve(hr), resolve(ar)
|
| 263 |
+
if not home or not away:
|
| 264 |
+
continue
|
| 265 |
+
gd = game.get("game_date", game.get("date", ""))[:10]
|
| 266 |
+
hr_ = team_results[home]
|
| 267 |
+
ar_ = team_results[away]
|
| 268 |
+
|
| 269 |
+
if len(hr_) < 5 or len(ar_) < 5:
|
| 270 |
+
team_results[home].append((gd, hs > as_, hs - as_, away, hs, as_))
|
| 271 |
+
team_results[away].append((gd, as_ > hs, as_ - hs, home, as_, hs))
|
| 272 |
+
team_last[home] = gd
|
| 273 |
+
team_last[away] = gd
|
| 274 |
+
K = 20
|
| 275 |
+
exp_h = 1 / (1 + 10 ** ((team_elo[away] - team_elo[home] - 50) / 400))
|
| 276 |
+
team_elo[home] += K * ((1 if hs > as_ else 0) - exp_h)
|
| 277 |
+
team_elo[away] += K * ((0 if hs > as_ else 1) - (1 - exp_h))
|
| 278 |
+
continue
|
| 279 |
+
|
| 280 |
+
def wp(r, n):
|
| 281 |
+
s = r[-n:]
|
| 282 |
+
return sum(1 for x in s if x[1]) / len(s) if s else 0.5
|
| 283 |
+
|
| 284 |
+
def pd(r, n):
|
| 285 |
+
s = r[-n:]
|
| 286 |
+
return sum(x[2] for x in s) / len(s) if s else 0.0
|
| 287 |
+
|
| 288 |
+
def ppg(r, n):
|
| 289 |
+
s = r[-n:]
|
| 290 |
+
return sum(x[4] for x in s) / len(s) if s else 100.0
|
| 291 |
+
|
| 292 |
+
def papg(r, n):
|
| 293 |
+
s = r[-n:]
|
| 294 |
+
return sum(x[5] for x in s) / len(s) if s else 100.0
|
| 295 |
+
|
| 296 |
+
def strk(r):
|
| 297 |
+
if not r: return 0
|
| 298 |
+
s, l = 0, r[-1][1]
|
| 299 |
+
for x in reversed(r):
|
| 300 |
+
if x[1] == l:
|
| 301 |
+
s += 1
|
| 302 |
+
else:
|
| 303 |
+
break
|
| 304 |
+
return s if l else -s
|
| 305 |
+
|
| 306 |
+
def close_pct(r, n):
|
| 307 |
+
s = r[-n:]
|
| 308 |
+
return sum(1 for x in s if abs(x[2]) <= 5) / len(s) if s else 0.5
|
| 309 |
+
|
| 310 |
+
def blowout_pct(r, n):
|
| 311 |
+
s = r[-n:]
|
| 312 |
+
return sum(1 for x in s if abs(x[2]) >= 15) / len(s) if s else 0.0
|
| 313 |
+
|
| 314 |
+
def consistency(r, n):
|
| 315 |
+
s = r[-n:]
|
| 316 |
+
if len(s) < 3: return 0.0
|
| 317 |
+
m = [x[2] for x in s]
|
| 318 |
+
avg = sum(m) / len(m)
|
| 319 |
+
return (sum((v - avg) ** 2 for v in m) / len(m)) ** 0.5
|
| 320 |
+
|
| 321 |
+
def rest(t):
|
| 322 |
+
last = team_last.get(t)
|
| 323 |
+
if not last or not gd: return 3
|
| 324 |
+
try:
|
| 325 |
+
return max(0, (datetime.strptime(gd[:10], "%Y-%m-%d") - datetime.strptime(last[:10], "%Y-%m-%d")).days)
|
| 326 |
+
except Exception:
|
| 327 |
+
return 3
|
| 328 |
+
|
| 329 |
+
def sos(r, n=10):
|
| 330 |
+
rec = r[-n:]
|
| 331 |
+
if not rec: return 0.5
|
| 332 |
+
ops = [wp(team_results[x[3]], 82) for x in rec if team_results[x[3]]]
|
| 333 |
+
return sum(ops) / len(ops) if ops else 0.5
|
| 334 |
+
|
| 335 |
+
def travel_dist(r, team):
|
| 336 |
+
if not r: return 0
|
| 337 |
+
last_opp = r[-1][3]
|
| 338 |
+
if last_opp in ARENA_COORDS and team in ARENA_COORDS:
|
| 339 |
+
return haversine(*ARENA_COORDS[last_opp], *ARENA_COORDS[team])
|
| 340 |
+
return 0
|
| 341 |
+
|
| 342 |
+
h_rest, a_rest = rest(home), rest(away)
|
| 343 |
+
try:
|
| 344 |
+
dt = datetime.strptime(gd, "%Y-%m-%d")
|
| 345 |
+
month, dow = dt.month, dt.weekday()
|
| 346 |
+
except Exception:
|
| 347 |
+
month, dow = 1, 2
|
| 348 |
+
|
| 349 |
+
sp = max(0, min(1, (month - 10) / 7)) if month >= 10 else max(0, min(1, (month + 2) / 7))
|
| 350 |
+
|
| 351 |
+
row = []
|
| 352 |
+
names = []
|
| 353 |
+
|
| 354 |
+
# 1. ROLLING PERFORMANCE (96 features)
|
| 355 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 356 |
+
for w in WINDOWS:
|
| 357 |
+
row.extend([wp(tr, w), pd(tr, w), ppg(tr, w), papg(tr, w),
|
| 358 |
+
ppg(tr, w) - papg(tr, w), close_pct(tr, w), blowout_pct(tr, w),
|
| 359 |
+
ppg(tr, w) + papg(tr, w)])
|
| 360 |
+
if first:
|
| 361 |
+
names.extend([f"{prefix}_wp{w}", f"{prefix}_pd{w}", f"{prefix}_ppg{w}",
|
| 362 |
+
f"{prefix}_papg{w}", f"{prefix}_margin{w}", f"{prefix}_close{w}",
|
| 363 |
+
f"{prefix}_blowout{w}", f"{prefix}_ou{w}"])
|
| 364 |
+
|
| 365 |
+
# 2. MOMENTUM (16 features)
|
| 366 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 367 |
+
row.extend([strk(tr), abs(strk(tr)),
|
| 368 |
+
wp(tr, 5) - wp(tr, 82), wp(tr, 3) - wp(tr, 10),
|
| 369 |
+
ppg(tr, 5) - ppg(tr, 20), papg(tr, 5) - papg(tr, 20),
|
| 370 |
+
consistency(tr, 10), consistency(tr, 5)])
|
| 371 |
+
if first:
|
| 372 |
+
names.extend([f"{prefix}_streak", f"{prefix}_streak_abs",
|
| 373 |
+
f"{prefix}_form5v82", f"{prefix}_form3v10",
|
| 374 |
+
f"{prefix}_scoring_trend", f"{prefix}_defense_trend",
|
| 375 |
+
f"{prefix}_consistency10", f"{prefix}_consistency5"])
|
| 376 |
+
|
| 377 |
+
# 3. REST & SCHEDULE (16 features)
|
| 378 |
+
h_travel = travel_dist(hr_, home)
|
| 379 |
+
a_travel = travel_dist(ar_, away)
|
| 380 |
+
row.extend([
|
| 381 |
+
min(h_rest, 7), min(a_rest, 7), h_rest - a_rest,
|
| 382 |
+
1.0 if h_rest <= 1 else 0.0, 1.0 if a_rest <= 1 else 0.0,
|
| 383 |
+
h_travel / 1000, a_travel / 1000, (h_travel - a_travel) / 1000,
|
| 384 |
+
ARENA_ALTITUDE.get(home, 500) / 5280, ARENA_ALTITUDE.get(away, 500) / 5280,
|
| 385 |
+
(ARENA_ALTITUDE.get(home, 500) - ARENA_ALTITUDE.get(away, 500)) / 5280,
|
| 386 |
+
abs(TIMEZONE_ET.get(home, 0) - TIMEZONE_ET.get(away, 0)),
|
| 387 |
+
0, 0, 0, 0,
|
| 388 |
+
])
|
| 389 |
+
if first:
|
| 390 |
+
names.extend(["h_rest", "a_rest", "rest_adv", "h_b2b", "a_b2b",
|
| 391 |
+
"h_travel", "a_travel", "travel_adv",
|
| 392 |
+
"h_altitude", "a_altitude", "altitude_delta",
|
| 393 |
+
"tz_shift", "h_games_7d", "a_games_7d", "sched_density", "pad1"])
|
| 394 |
+
|
| 395 |
+
# 4. OPPONENT-ADJUSTED (12 features)
|
| 396 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 397 |
+
s5 = sos(tr, 5)
|
| 398 |
+
s10 = sos(tr, 10)
|
| 399 |
+
ss = sos(tr, 82)
|
| 400 |
+
wp_above = sum(1 for r in tr if wp(team_results[r[3]], 82) > 0.5 and r[1]) / max(
|
| 401 |
+
sum(1 for r in tr if wp(team_results[r[3]], 82) > 0.5), 1)
|
| 402 |
+
wp_below = sum(1 for r in tr if wp(team_results[r[3]], 82) <= 0.5 and r[1]) / max(
|
| 403 |
+
sum(1 for r in tr if wp(team_results[r[3]], 82) <= 0.5), 1)
|
| 404 |
+
row.extend([s5, s10, ss, wp_above, wp_below, 0])
|
| 405 |
+
if first:
|
| 406 |
+
names.extend([f"{prefix}_sos5", f"{prefix}_sos10", f"{prefix}_sos_season",
|
| 407 |
+
f"{prefix}_wp_above500", f"{prefix}_wp_below500", f"{prefix}_margin_quality"])
|
| 408 |
+
|
| 409 |
+
# 5. MATCHUP & ELO (12 features)
|
| 410 |
+
row.extend([
|
| 411 |
+
wp(hr_, 10) - wp(ar_, 10), pd(hr_, 10) - pd(ar_, 10),
|
| 412 |
+
ppg(hr_, 10) - papg(ar_, 10), ppg(ar_, 10) - papg(hr_, 10),
|
| 413 |
+
abs(ppg(hr_, 10) + papg(hr_, 10) - ppg(ar_, 10) - papg(ar_, 10)),
|
| 414 |
+
consistency(hr_, 10) - consistency(ar_, 10),
|
| 415 |
+
team_elo[home], team_elo[away], team_elo[home] - team_elo[away] + 50,
|
| 416 |
+
(team_elo[home] - 1500) / 100, (team_elo[away] - 1500) / 100,
|
| 417 |
+
(team_elo[home] - team_elo[away]) / 100,
|
| 418 |
+
])
|
| 419 |
+
if first:
|
| 420 |
+
names.extend(["rel_strength", "rel_pd", "off_matchup", "def_matchup",
|
| 421 |
+
"tempo_diff", "consistency_edge",
|
| 422 |
+
"elo_home", "elo_away", "elo_diff",
|
| 423 |
+
"elo_home_norm", "elo_away_norm", "elo_diff_norm"])
|
| 424 |
+
|
| 425 |
+
# 6. CONTEXT (12 features)
|
| 426 |
+
row.extend([
|
| 427 |
+
1.0, sp, math.sin(2 * math.pi * month / 12), math.cos(2 * math.pi * month / 12),
|
| 428 |
+
dow / 6.0, 1.0 if dow >= 5 else 0.0,
|
| 429 |
+
min(len(hr_), 82) / 82.0, min(len(ar_), 82) / 82.0,
|
| 430 |
+
wp(hr_, 82) + wp(ar_, 82), wp(hr_, 82) - wp(ar_, 82),
|
| 431 |
+
1.0 if wp(hr_, 82) > 0.5 and wp(ar_, 82) > 0.5 else 0.0,
|
| 432 |
+
ppg(hr_, 10) + ppg(ar_, 10),
|
| 433 |
+
])
|
| 434 |
+
if first:
|
| 435 |
+
names.extend(["home_court", "season_phase", "month_sin", "month_cos",
|
| 436 |
+
"day_of_week", "is_weekend", "h_games_pct", "a_games_pct",
|
| 437 |
+
"combined_wp", "wp_diff", "playoff_race", "expected_total"])
|
| 438 |
+
|
| 439 |
+
# 7. CROSS-WINDOW MOMENTUM (20 features) — trend acceleration
|
| 440 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 441 |
+
# Short vs long momentum (5 vs 20)
|
| 442 |
+
wp_accel = wp(tr, 3) - 2 * wp(tr, 10) + wp(tr, 20) if len(tr) >= 20 else 0.0
|
| 443 |
+
pd_accel = pd(tr, 3) - 2 * pd(tr, 10) + pd(tr, 20) if len(tr) >= 20 else 0.0
|
| 444 |
+
# Pythagorean expected win rate (Bill James)
|
| 445 |
+
pts_for = sum(x[4] for x in tr[-20:]) if len(tr) >= 5 else 100
|
| 446 |
+
pts_against = sum(x[5] for x in tr[-20:]) if len(tr) >= 5 else 100
|
| 447 |
+
pyth_exp = pts_for ** 13.91 / max(1, pts_for ** 13.91 + pts_against ** 13.91) if pts_for > 0 else 0.5
|
| 448 |
+
# Scoring volatility
|
| 449 |
+
pts_list = [x[4] for x in tr[-10:]] if len(tr) >= 5 else [100]
|
| 450 |
+
pts_vol = (sum((p - sum(pts_list)/len(pts_list))**2 for p in pts_list) / len(pts_list)) ** 0.5 if len(pts_list) > 1 else 0
|
| 451 |
+
# Home/away specific win rates
|
| 452 |
+
home_games = [x for x in tr if x[3] != home] if prefix == "h" else [x for x in tr if x[3] != away]
|
| 453 |
+
ha_wp = sum(1 for x in home_games[-20:] if x[1]) / max(len(home_games[-20:]), 1)
|
| 454 |
+
# Opponent quality of recent wins
|
| 455 |
+
recent_wins = [x for x in tr[-10:] if x[1]]
|
| 456 |
+
win_quality = sum(wp(team_results[x[3]], 82) for x in recent_wins) / max(len(recent_wins), 1) if recent_wins else 0.5
|
| 457 |
+
# Margin trend (linear slope over last 10 games)
|
| 458 |
+
margins_10 = [x[2] for x in tr[-10:]] if len(tr) >= 5 else [0]
|
| 459 |
+
if len(margins_10) >= 3:
|
| 460 |
+
x_vals = list(range(len(margins_10)))
|
| 461 |
+
x_mean = sum(x_vals) / len(x_vals)
|
| 462 |
+
y_mean = sum(margins_10) / len(margins_10)
|
| 463 |
+
num = sum((x - x_mean) * (y - y_mean) for x, y in zip(x_vals, margins_10))
|
| 464 |
+
den = sum((x - x_mean) ** 2 for x in x_vals)
|
| 465 |
+
margin_slope = num / den if den > 0 else 0.0
|
| 466 |
+
else:
|
| 467 |
+
margin_slope = 0.0
|
| 468 |
+
row.extend([
|
| 469 |
+
wp(tr, 5) - wp(tr, 20) if len(tr) >= 20 else 0.0,
|
| 470 |
+
wp_accel, pd_accel, pyth_exp,
|
| 471 |
+
pts_vol / 10.0, # normalized
|
| 472 |
+
ha_wp, win_quality,
|
| 473 |
+
margin_slope,
|
| 474 |
+
ppg(tr, 3) / max(ppg(tr, 20), 1), # recent scoring ratio
|
| 475 |
+
papg(tr, 3) / max(papg(tr, 20), 1), # recent defense ratio
|
| 476 |
+
])
|
| 477 |
+
if first:
|
| 478 |
+
names.extend([f"{prefix}_wp5v20", f"{prefix}_wp_accel", f"{prefix}_pd_accel",
|
| 479 |
+
f"{prefix}_pyth_exp", f"{prefix}_pts_vol",
|
| 480 |
+
f"{prefix}_location_wp", f"{prefix}_win_quality",
|
| 481 |
+
f"{prefix}_margin_slope", f"{prefix}_off_ratio", f"{prefix}_def_ratio"])
|
| 482 |
+
|
| 483 |
+
# 8. INTERACTION FEATURES (12 features) — key cross-terms
|
| 484 |
+
elo_d = team_elo[home] - team_elo[away] + 50
|
| 485 |
+
rest_adv = h_rest - a_rest
|
| 486 |
+
wp_d = wp(hr_, 10) - wp(ar_, 10)
|
| 487 |
+
row.extend([
|
| 488 |
+
elo_d * rest_adv / 10.0, # elo × rest interaction
|
| 489 |
+
wp_d * rest_adv / 3.0, # form × rest interaction
|
| 490 |
+
elo_d * (1 if h_rest <= 1 else 0), # elo × b2b penalty
|
| 491 |
+
wp_d ** 2, # squared wp diff (nonlinearity)
|
| 492 |
+
elo_d ** 2 / 10000.0, # squared elo diff
|
| 493 |
+
(ppg(hr_, 10) - papg(ar_, 10)) * (ppg(ar_, 10) - papg(hr_, 10)), # off×def interaction
|
| 494 |
+
consistency(hr_, 10) * consistency(ar_, 10) / 100.0, # consistency product
|
| 495 |
+
wp(hr_, 82) * wp(ar_, 82), # season quality product
|
| 496 |
+
(wp(hr_, 5) - wp(hr_, 20)) * (wp(ar_, 5) - wp(ar_, 20)), # momentum alignment
|
| 497 |
+
abs(ppg(hr_, 10) + papg(hr_, 10) - ppg(ar_, 10) - papg(ar_, 10)) * elo_d / 1000.0, # tempo×elo
|
| 498 |
+
(1.0 if wp(hr_, 82) > 0.6 else 0.0) * (1.0 if wp(ar_, 82) < 0.4 else 0.0), # mismatch flag
|
| 499 |
+
float(h_rest >= 3 and a_rest <= 1), # rest mismatch flag
|
| 500 |
+
])
|
| 501 |
+
if first:
|
| 502 |
+
names.extend(["elo_rest_interact", "form_rest_interact", "elo_b2b_penalty",
|
| 503 |
+
"wp_diff_sq", "elo_diff_sq", "off_def_interact",
|
| 504 |
+
"consistency_product", "quality_product", "momentum_align",
|
| 505 |
+
"tempo_elo_interact", "mismatch_flag", "rest_mismatch_flag"])
|
| 506 |
+
|
| 507 |
+
# 9. NEW HIGH-IMPACT FEATURES (50 features, windows [5, 10])
|
| 508 |
+
NEW_WINDOWS = [5, 10]
|
| 509 |
+
|
| 510 |
+
# Helper: home/away split win% (home team plays at home, away team plays away)
|
| 511 |
+
def home_split_wp(r, n, is_home_team):
|
| 512 |
+
"""Win% for home-only or away-only games over last n."""
|
| 513 |
+
if is_home_team:
|
| 514 |
+
# home team's results when they were the home team (opponent is different city)
|
| 515 |
+
loc_games = [x for x in r if x[3] != home][-n:]
|
| 516 |
+
else:
|
| 517 |
+
loc_games = [x for x in r if x[3] != away][-n:]
|
| 518 |
+
if not loc_games:
|
| 519 |
+
return wp(r, n) # fallback to overall
|
| 520 |
+
return sum(1 for x in loc_games if x[1]) / len(loc_games)
|
| 521 |
+
|
| 522 |
+
def away_split_wp(r, n, is_home_team):
|
| 523 |
+
"""Win% for away-only games over last n."""
|
| 524 |
+
if is_home_team:
|
| 525 |
+
loc_games = [x for x in r if x[3] == home][-n:]
|
| 526 |
+
else:
|
| 527 |
+
loc_games = [x for x in r if x[3] == away][-n:]
|
| 528 |
+
if not loc_games:
|
| 529 |
+
return wp(r, n)
|
| 530 |
+
return sum(1 for x in loc_games if x[1]) / len(loc_games)
|
| 531 |
+
|
| 532 |
+
def net_rating(r, n):
|
| 533 |
+
"""Net points per game over window (proxy for net rating)."""
|
| 534 |
+
s = r[-n:]
|
| 535 |
+
if not s:
|
| 536 |
+
return 0.0
|
| 537 |
+
return sum(x[4] - x[5] for x in s) / len(s)
|
| 538 |
+
|
| 539 |
+
def pace_proxy(r, n):
|
| 540 |
+
"""Approximate pace as total points per game (proxy when possession data absent)."""
|
| 541 |
+
s = r[-n:]
|
| 542 |
+
if not s:
|
| 543 |
+
return 200.0
|
| 544 |
+
return sum(x[4] + x[5] for x in s) / len(s)
|
| 545 |
+
|
| 546 |
+
def h2h_wp(hr, ar, n):
|
| 547 |
+
"""Head-to-head win% for home team vs this specific away team over last n meetings."""
|
| 548 |
+
meetings = [x for x in hr if x[3] == away][-n:]
|
| 549 |
+
if not meetings:
|
| 550 |
+
return 0.5
|
| 551 |
+
return sum(1 for x in meetings if x[1]) / len(meetings)
|
| 552 |
+
|
| 553 |
+
def sos_window(r, n):
|
| 554 |
+
"""Average opponent win% over last n games (Strength of Schedule)."""
|
| 555 |
+
rec = r[-n:]
|
| 556 |
+
if not rec:
|
| 557 |
+
return 0.5
|
| 558 |
+
ops = [wp(team_results[x[3]], 82) for x in rec if team_results[x[3]]]
|
| 559 |
+
return sum(ops) / len(ops) if ops else 0.5
|
| 560 |
+
|
| 561 |
+
# 9a. Net Rating (windows 5, 10) — 4 features
|
| 562 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 563 |
+
for w in NEW_WINDOWS:
|
| 564 |
+
row.append(net_rating(tr, w))
|
| 565 |
+
if first:
|
| 566 |
+
names.append(f"{prefix}_net_rating{w}")
|
| 567 |
+
|
| 568 |
+
# 9b. Pace proxy (windows 5, 10) — 4 features
|
| 569 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 570 |
+
for w in NEW_WINDOWS:
|
| 571 |
+
row.append(pace_proxy(tr, w))
|
| 572 |
+
if first:
|
| 573 |
+
names.append(f"{prefix}_pace{w}")
|
| 574 |
+
|
| 575 |
+
# 9c. Rest days (already exists as h_rest/a_rest, add explicit named vars for clarity)
|
| 576 |
+
# These are already in section 3 above; skip to avoid duplication.
|
| 577 |
+
|
| 578 |
+
# 9d. Home/Away Win% Split (windows 5, 10) — 4 features each side = 8 features
|
| 579 |
+
for w in NEW_WINDOWS:
|
| 580 |
+
row.append(home_split_wp(hr_, w, is_home_team=True)) # h home-venue wp
|
| 581 |
+
row.append(away_split_wp(ar_, w, is_home_team=False)) # a away-venue wp
|
| 582 |
+
if first:
|
| 583 |
+
names.append(f"h_home_wp{w}")
|
| 584 |
+
names.append(f"a_away_wp{w}")
|
| 585 |
+
|
| 586 |
+
# 9e. Matchup H2H record (windows 5, 10) — 2 features
|
| 587 |
+
for w in NEW_WINDOWS:
|
| 588 |
+
row.append(h2h_wp(hr_, ar_, w))
|
| 589 |
+
if first:
|
| 590 |
+
names.append(f"h_h2h_wp{w}")
|
| 591 |
+
|
| 592 |
+
# 9f. Strength of Schedule windows 5, 10 (distinct from existing sos5/sos10 in sec 4)
|
| 593 |
+
# sec 4 already has h_sos5, h_sos10 — skip to avoid duplication.
|
| 594 |
+
|
| 595 |
+
# 9g. Streak type: signed streak (positive=wins, negative=losses) — 2 features
|
| 596 |
+
# (strk() already included in section 2 as h_streak/a_streak; skip duplicate.)
|
| 597 |
+
|
| 598 |
+
# 9h. Pace × Net Rating interaction — 4 features (home + away, windows 5 and 10)
|
| 599 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 600 |
+
for w in NEW_WINDOWS:
|
| 601 |
+
p = pace_proxy(tr, w)
|
| 602 |
+
n_r = net_rating(tr, w)
|
| 603 |
+
row.append((p * n_r) / 1000.0) # scaled
|
| 604 |
+
if first:
|
| 605 |
+
names.append(f"{prefix}_pace_net_interact{w}")
|
| 606 |
+
|
| 607 |
+
# 9i. Pythagorean-adjusted net rating (per-100-possessions approximation) — 4 features
|
| 608 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 609 |
+
for w in NEW_WINDOWS:
|
| 610 |
+
s = tr[-w:]
|
| 611 |
+
if s:
|
| 612 |
+
total_pts_for = sum(x[4] for x in s)
|
| 613 |
+
total_pts_ag = sum(x[5] for x in s)
|
| 614 |
+
n_games = len(s)
|
| 615 |
+
avg_pace = (total_pts_for + total_pts_ag) / max(n_games, 1)
|
| 616 |
+
# net per 100 possessions approximation
|
| 617 |
+
net_per100 = ((total_pts_for - total_pts_ag) / max(n_games, 1)) / max(avg_pace / 100.0, 1.0)
|
| 618 |
+
else:
|
| 619 |
+
net_per100 = 0.0
|
| 620 |
+
row.append(net_per100)
|
| 621 |
+
if first:
|
| 622 |
+
names.append(f"{prefix}_net_per100_{w}")
|
| 623 |
+
|
| 624 |
+
# 9j. Recent opponent quality (win% of opponents faced) — 4 features (windows 5, 10)
|
| 625 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 626 |
+
for w in NEW_WINDOWS:
|
| 627 |
+
row.append(sos_window(tr, w))
|
| 628 |
+
if first:
|
| 629 |
+
names.append(f"{prefix}_opp_quality{w}")
|
| 630 |
+
|
| 631 |
+
# ── SECTION 10: EXPONENTIALLY-WEIGHTED MOMENTUM FEATURES (~28 features) ──
|
| 632 |
+
# EWM uses manual exponential decay (no pandas needed) for each team's history.
|
| 633 |
+
# Halflife h means the weight of a game h games ago is 0.5x the weight of the current.
|
| 634 |
+
# alpha = 1 - exp(-ln(2) / halflife) => older games decay exponentially.
|
| 635 |
+
|
| 636 |
+
def ewm_win(r, halflife):
|
| 637 |
+
"""EWM of wins (0/1) with given halflife in games."""
|
| 638 |
+
s = [x[1] for x in r]
|
| 639 |
+
if not s:
|
| 640 |
+
return 0.5
|
| 641 |
+
alpha = 1.0 - math.exp(-math.log(2) / max(halflife, 0.5))
|
| 642 |
+
val, w_sum = 0.0, 0.0
|
| 643 |
+
for i, v in enumerate(s):
|
| 644 |
+
w = (1 - alpha) ** (len(s) - 1 - i)
|
| 645 |
+
val += w * float(v)
|
| 646 |
+
w_sum += w
|
| 647 |
+
return val / w_sum if w_sum > 0 else 0.5
|
| 648 |
+
|
| 649 |
+
def ewm_pd(r, halflife):
|
| 650 |
+
"""EWM of point differentials with given halflife."""
|
| 651 |
+
s = [x[2] for x in r]
|
| 652 |
+
if not s:
|
| 653 |
+
return 0.0
|
| 654 |
+
alpha = 1.0 - math.exp(-math.log(2) / max(halflife, 0.5))
|
| 655 |
+
val, w_sum = 0.0, 0.0
|
| 656 |
+
for i, v in enumerate(s):
|
| 657 |
+
w = (1 - alpha) ** (len(s) - 1 - i)
|
| 658 |
+
val += w * v
|
| 659 |
+
w_sum += w
|
| 660 |
+
return val / w_sum if w_sum > 0 else 0.0
|
| 661 |
+
|
| 662 |
+
def ewm_ppg(r, halflife):
|
| 663 |
+
"""EWM of points scored per game."""
|
| 664 |
+
s = [x[4] for x in r]
|
| 665 |
+
if not s:
|
| 666 |
+
return 100.0
|
| 667 |
+
alpha = 1.0 - math.exp(-math.log(2) / max(halflife, 0.5))
|
| 668 |
+
val, w_sum = 0.0, 0.0
|
| 669 |
+
for i, v in enumerate(s):
|
| 670 |
+
w = (1 - alpha) ** (len(s) - 1 - i)
|
| 671 |
+
val += w * v
|
| 672 |
+
w_sum += w
|
| 673 |
+
return val / w_sum if w_sum > 0 else 100.0
|
| 674 |
+
|
| 675 |
+
def ewm_papg(r, halflife):
|
| 676 |
+
"""EWM of opponent points per game (defensive rating proxy)."""
|
| 677 |
+
s = [x[5] for x in r]
|
| 678 |
+
if not s:
|
| 679 |
+
return 100.0
|
| 680 |
+
alpha = 1.0 - math.exp(-math.log(2) / max(halflife, 0.5))
|
| 681 |
+
val, w_sum = 0.0, 0.0
|
| 682 |
+
for i, v in enumerate(s):
|
| 683 |
+
w = (1 - alpha) ** (len(s) - 1 - i)
|
| 684 |
+
val += w * v
|
| 685 |
+
w_sum += w
|
| 686 |
+
return val / w_sum if w_sum > 0 else 100.0
|
| 687 |
+
|
| 688 |
+
def streak_decay_score(r):
|
| 689 |
+
"""current_streak x (1 / (1 + games_since_last_loss)).
|
| 690 |
+
Captures both streak length and recency of last loss."""
|
| 691 |
+
if not r:
|
| 692 |
+
return 0.0
|
| 693 |
+
cur_streak = 0
|
| 694 |
+
last_result = r[-1][1]
|
| 695 |
+
for x in reversed(r):
|
| 696 |
+
if x[1] == last_result:
|
| 697 |
+
cur_streak += 1
|
| 698 |
+
else:
|
| 699 |
+
break
|
| 700 |
+
if not last_result:
|
| 701 |
+
return -float(cur_streak) # losing streak: negative
|
| 702 |
+
# Count consecutive games back since last loss (= win streak length)
|
| 703 |
+
games_since_loss = 0
|
| 704 |
+
for x in reversed(r):
|
| 705 |
+
if not x[1]:
|
| 706 |
+
break
|
| 707 |
+
games_since_loss += 1
|
| 708 |
+
return cur_streak * (1.0 / (1 + games_since_loss))
|
| 709 |
+
|
| 710 |
+
def fatigue_index(r, n=5):
|
| 711 |
+
"""Sum of (1/rest_days) for last n inter-game gaps — high = compressed schedule."""
|
| 712 |
+
recent = r[-n:]
|
| 713 |
+
if len(recent) < 2:
|
| 714 |
+
return 0.0
|
| 715 |
+
total = 0.0
|
| 716 |
+
for i in range(1, len(recent)):
|
| 717 |
+
try:
|
| 718 |
+
d1 = datetime.strptime(recent[i - 1][0][:10], "%Y-%m-%d")
|
| 719 |
+
d2 = datetime.strptime(recent[i][0][:10], "%Y-%m-%d")
|
| 720 |
+
gap = max(1, abs((d2 - d1).days))
|
| 721 |
+
total += 1.0 / gap
|
| 722 |
+
except Exception:
|
| 723 |
+
total += 0.5 # fallback: assume 2-day gap
|
| 724 |
+
return total
|
| 725 |
+
|
| 726 |
+
def b2b_delta(r, metric_idx=2):
|
| 727 |
+
"""B2B performance delta: avg metric in B2B games minus avg in normal-rest games.
|
| 728 |
+
B2B = previous game was <= 1 day ago."""
|
| 729 |
+
b2b_vals, normal_vals = [], []
|
| 730 |
+
for i in range(1, len(r)):
|
| 731 |
+
try:
|
| 732 |
+
d1 = datetime.strptime(r[i - 1][0][:10], "%Y-%m-%d")
|
| 733 |
+
d2 = datetime.strptime(r[i][0][:10], "%Y-%m-%d")
|
| 734 |
+
gap = abs((d2 - d1).days)
|
| 735 |
+
except Exception:
|
| 736 |
+
gap = 2
|
| 737 |
+
val = r[i][metric_idx]
|
| 738 |
+
if gap <= 1:
|
| 739 |
+
b2b_vals.append(val)
|
| 740 |
+
else:
|
| 741 |
+
normal_vals.append(val)
|
| 742 |
+
b2b_avg = sum(b2b_vals) / len(b2b_vals) if b2b_vals else 0.0
|
| 743 |
+
normal_avg = sum(normal_vals) / len(normal_vals) if normal_vals else 0.0
|
| 744 |
+
return b2b_avg - normal_avg
|
| 745 |
+
|
| 746 |
+
def travel_burden(r, n=7):
|
| 747 |
+
"""Count unique opponents (proxy for unique cities visited) in last n games.
|
| 748 |
+
More unique opponents correlates with more travel across the schedule."""
|
| 749 |
+
recent = r[-n:]
|
| 750 |
+
if not recent:
|
| 751 |
+
return 0
|
| 752 |
+
return len({x[3] for x in recent})
|
| 753 |
+
|
| 754 |
+
# 10a. EWM Win Probability — halflives [3, 5, 10] x 2 teams = 6 features
|
| 755 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 756 |
+
for hl in [3, 5, 10]:
|
| 757 |
+
row.append(ewm_win(tr, hl))
|
| 758 |
+
if first:
|
| 759 |
+
names.append(f"{prefix}_ewm_win_hl{hl}")
|
| 760 |
+
|
| 761 |
+
# 10b. EWM Point Differential — halflives [3, 5, 10] x 2 teams = 6 features
|
| 762 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 763 |
+
for hl in [3, 5, 10]:
|
| 764 |
+
row.append(ewm_pd(tr, hl) / 10.0) # normalize: typical margins ~0–20 pts
|
| 765 |
+
if first:
|
| 766 |
+
names.append(f"{prefix}_ewm_pd_hl{hl}")
|
| 767 |
+
|
| 768 |
+
# 10c. EWM Offensive Rating (halflife=5) — 2 features
|
| 769 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 770 |
+
row.append(ewm_ppg(tr, 5) / 100.0) # normalize to ~1.0 range
|
| 771 |
+
if first:
|
| 772 |
+
names.append(f"{prefix}_ewm_off_hl5")
|
| 773 |
+
|
| 774 |
+
# 10d. EWM Defensive Rating (halflife=5) — 2 features
|
| 775 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 776 |
+
row.append(ewm_papg(tr, 5) / 100.0)
|
| 777 |
+
if first:
|
| 778 |
+
names.append(f"{prefix}_ewm_def_hl5")
|
| 779 |
+
|
| 780 |
+
# 10e. Streak Decay Score — 2 features
|
| 781 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 782 |
+
row.append(streak_decay_score(tr))
|
| 783 |
+
if first:
|
| 784 |
+
names.append(f"{prefix}_streak_decay")
|
| 785 |
+
|
| 786 |
+
# 10f. Fatigue Index (last 5 games) — 2 features
|
| 787 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 788 |
+
row.append(fatigue_index(tr, n=5))
|
| 789 |
+
if first:
|
| 790 |
+
names.append(f"{prefix}_fatigue_idx")
|
| 791 |
+
|
| 792 |
+
# 10g. B2B Performance Delta (point margin) — 2 features
|
| 793 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 794 |
+
row.append(b2b_delta(tr, metric_idx=2) / 10.0) # normalized margin delta
|
| 795 |
+
if first:
|
| 796 |
+
names.append(f"{prefix}_b2b_margin_delta")
|
| 797 |
+
|
| 798 |
+
# 10h. Travel Burden (unique cities proxy over last 7 games) — 2 features
|
| 799 |
+
for prefix, tr in [("h", hr_), ("a", ar_)]:
|
| 800 |
+
row.append(float(travel_burden(tr, n=7)) / 7.0) # normalize to [0, 1]
|
| 801 |
+
if first:
|
| 802 |
+
names.append(f"{prefix}_travel_burden7")
|
| 803 |
+
|
| 804 |
+
# 10i. Cross-team EWM interaction features — 4 features
|
| 805 |
+
row.append(ewm_win(hr_, 3) - ewm_win(ar_, 3)) # home vs away momentum (hl=3)
|
| 806 |
+
row.append(ewm_win(hr_, 5) - ewm_win(ar_, 5)) # home vs away momentum (hl=5)
|
| 807 |
+
row.append((ewm_pd(hr_, 5) - ewm_pd(ar_, 5)) / 10.0) # relative margin quality (hl=5)
|
| 808 |
+
row.append((ewm_ppg(hr_, 5) - ewm_papg(ar_, 5)) / 100.0) # home offense vs away defense
|
| 809 |
+
if first:
|
| 810 |
+
names.extend(["ewm_win_diff_hl3", "ewm_win_diff_hl5",
|
| 811 |
+
"ewm_pd_diff_hl5", "ewm_off_vs_def_hl5"])
|
| 812 |
+
|
| 813 |
+
X.append(row)
|
| 814 |
+
y.append(1 if hs > as_ else 0)
|
| 815 |
+
if first:
|
| 816 |
+
feature_names = names
|
| 817 |
+
first = False
|
| 818 |
+
|
| 819 |
+
team_results[home].append((gd, hs > as_, hs - as_, away, hs, as_))
|
| 820 |
+
team_results[away].append((gd, as_ > hs, as_ - hs, home, as_, hs))
|
| 821 |
+
team_last[home] = gd
|
| 822 |
+
team_last[away] = gd
|
| 823 |
+
K = 20
|
| 824 |
+
exp_h = 1 / (1 + 10 ** ((team_elo[away] - team_elo[home] - 50) / 400))
|
| 825 |
+
team_elo[home] += K * ((1 if hs > as_ else 0) - exp_h)
|
| 826 |
+
team_elo[away] += K * ((0 if hs > as_ else 1) - (1 - exp_h))
|
| 827 |
+
|
| 828 |
+
X = np.nan_to_num(np.array(X, dtype=np.float64))
|
| 829 |
+
y = np.array(y, dtype=np.int32)
|
| 830 |
+
return X, y, feature_names
|
| 831 |
+
|
| 832 |
+
|
| 833 |
+
# ═══════════════════════════════════════════════════════════
|
| 834 |
+
# SECTION 3: INDIVIDUAL (feature mask + hyperparameters)
|
| 835 |
+
# ═══════════════════════════════════════════════════════════
|
| 836 |
+
|
| 837 |
+
class Individual:
|
| 838 |
+
"""One model configuration: feature selection mask + hyperparameters."""
|
| 839 |
+
|
| 840 |
+
def __init__(self, n_features, target=100, model_type=None):
|
| 841 |
+
prob = target / max(n_features, 1)
|
| 842 |
+
self.features = [1 if random.random() < prob else 0 for _ in range(n_features)]
|
| 843 |
+
self.hyperparams = {
|
| 844 |
+
"n_estimators": random.randint(100, 600),
|
| 845 |
+
"max_depth": random.randint(3, 10),
|
| 846 |
+
"learning_rate": 10 ** random.uniform(-2.5, -0.5),
|
| 847 |
+
"subsample": random.uniform(0.5, 1.0),
|
| 848 |
+
"colsample_bytree": random.uniform(0.3, 1.0),
|
| 849 |
+
"min_child_weight": random.randint(1, 15),
|
| 850 |
+
"reg_alpha": 10 ** random.uniform(-6, 1),
|
| 851 |
+
"reg_lambda": 10 ** random.uniform(-6, 1),
|
| 852 |
+
"model_type": model_type or random.choice(GPU_MODEL_TYPES),
|
| 853 |
+
# venn_abers added (brain cycle 82, 2026-04-09): HF island best (pareto 0.21773) uses
|
| 854 |
+
# venn_abers; GPU Kaggle loop was missing this option entirely.
|
| 855 |
+
"calibration": random.choices(
|
| 856 |
+
["none", "sigmoid", "venn_abers", "beta", "isotonic", "isotonic_temporal"],
|
| 857 |
+
weights=[15, 12, 25, 20, 15, 13], k=1)[0],
|
| 858 |
+
# Neural net hyperparams
|
| 859 |
+
"nn_hidden_dims": random.choice([64, 128, 256]),
|
| 860 |
+
"nn_n_layers": random.randint(2, 4),
|
| 861 |
+
"nn_dropout": random.uniform(0.1, 0.5),
|
| 862 |
+
"nn_epochs": random.randint(20, 100),
|
| 863 |
+
"nn_batch_size": random.choice([32, 64, 128]),
|
| 864 |
+
}
|
| 865 |
+
self.fitness = {"brier": 1.0, "roi": 0.0, "sharpe": 0.0, "calibration": 1.0, "calibration_error": 1.0, "composite": 0.0}
|
| 866 |
+
self.pareto_rank = 999
|
| 867 |
+
self.crowding_dist = 0.0
|
| 868 |
+
self.island_id = -1
|
| 869 |
+
self.generation = 0
|
| 870 |
+
self.birth_generation = 0
|
| 871 |
+
self._enforce_feature_cap()
|
| 872 |
+
|
| 873 |
+
def selected_indices(self):
|
| 874 |
+
return [i for i, b in enumerate(self.features) if b]
|
| 875 |
+
|
| 876 |
+
def to_dict(self):
|
| 877 |
+
return {
|
| 878 |
+
"n_features": self.n_features,
|
| 879 |
+
"hyperparams": {k: v for k, v in self.hyperparams.items()},
|
| 880 |
+
"fitness": dict(self.fitness),
|
| 881 |
+
"generation": self.generation,
|
| 882 |
+
}
|
| 883 |
+
|
| 884 |
+
@staticmethod
|
| 885 |
+
def _hamming_distance(f1, f2):
|
| 886 |
+
"""Normalized Hamming distance between two binary feature masks (0.0 – 1.0)."""
|
| 887 |
+
n = len(f1)
|
| 888 |
+
if n == 0:
|
| 889 |
+
return 0.0
|
| 890 |
+
return sum(a != b for a, b in zip(f1, f2)) / n
|
| 891 |
+
|
| 892 |
+
@staticmethod
|
| 893 |
+
def crossover(p1, p2):
|
| 894 |
+
"""Crossover on features + blend hyperparams.
|
| 895 |
+
|
| 896 |
+
Crossover type is selected based on parent similarity:
|
| 897 |
+
- Parents very similar (Hamming < 0.1): uniform crossover.
|
| 898 |
+
Picks each bit independently, generating more variation between
|
| 899 |
+
nearly-identical individuals.
|
| 900 |
+
- Otherwise: classic two-point crossover.
|
| 901 |
+
"""
|
| 902 |
+
child = Individual.__new__(Individual)
|
| 903 |
+
n = len(p1.features)
|
| 904 |
+
parent_hamming = Individual._hamming_distance(p1.features, p2.features)
|
| 905 |
+
if parent_hamming < 0.1:
|
| 906 |
+
# Uniform crossover: each position drawn independently
|
| 907 |
+
child.features = [
|
| 908 |
+
p1.features[i] if random.random() < 0.5 else p2.features[i]
|
| 909 |
+
for i in range(n)
|
| 910 |
+
]
|
| 911 |
+
else:
|
| 912 |
+
pt1 = random.randint(0, n - 1)
|
| 913 |
+
pt2 = random.randint(pt1, n - 1)
|
| 914 |
+
child.features = p1.features[:pt1] + p2.features[pt1:pt2] + p1.features[pt2:]
|
| 915 |
+
|
| 916 |
+
child.hyperparams = {}
|
| 917 |
+
for key in p1.hyperparams:
|
| 918 |
+
if isinstance(p1.hyperparams[key], (int, float)):
|
| 919 |
+
w = random.random()
|
| 920 |
+
val = w * p1.hyperparams[key] + (1 - w) * p2.hyperparams[key]
|
| 921 |
+
if isinstance(p1.hyperparams[key], int):
|
| 922 |
+
val = int(round(val))
|
| 923 |
+
child.hyperparams[key] = val
|
| 924 |
+
else:
|
| 925 |
+
child.hyperparams[key] = random.choice([p1.hyperparams[key], p2.hyperparams[key]])
|
| 926 |
+
|
| 927 |
+
child.fitness = {"brier": 1.0, "roi": 0.0, "sharpe": 0.0, "calibration": 1.0, "calibration_error": 1.0, "composite": 0.0}
|
| 928 |
+
child.generation = max(p1.generation, p2.generation) + 1
|
| 929 |
+
child.birth_generation = child.generation
|
| 930 |
+
child.pareto_rank = 999
|
| 931 |
+
child.crowding_dist = 0.0
|
| 932 |
+
child.island_id = -1
|
| 933 |
+
child._enforce_feature_cap()
|
| 934 |
+
return child
|
| 935 |
+
|
| 936 |
+
MAX_FEATURES = 200 # Hard cap — individuals above this waste compute
|
| 937 |
+
|
| 938 |
+
def _enforce_feature_cap(self):
|
| 939 |
+
"""If feature count exceeds MAX_FEATURES, randomly drop excess features."""
|
| 940 |
+
selected = [i for i, b in enumerate(self.features) if b]
|
| 941 |
+
if len(selected) > self.MAX_FEATURES:
|
| 942 |
+
to_drop = random.sample(selected, len(selected) - self.MAX_FEATURES)
|
| 943 |
+
for idx in to_drop:
|
| 944 |
+
self.features[idx] = 0
|
| 945 |
+
self.n_features = sum(self.features)
|
| 946 |
+
|
| 947 |
+
def mutate(self, rate=0.03):
|
| 948 |
+
"""Mutate features and hyperparameters."""
|
| 949 |
+
for i in range(len(self.features)):
|
| 950 |
+
if random.random() < rate:
|
| 951 |
+
self.features[i] = 1 - self.features[i]
|
| 952 |
+
self._enforce_feature_cap()
|
| 953 |
+
if random.random() < 0.15:
|
| 954 |
+
self.hyperparams["n_estimators"] = max(50, self.hyperparams["n_estimators"] + random.randint(-100, 100))
|
| 955 |
+
if random.random() < 0.15:
|
| 956 |
+
self.hyperparams["max_depth"] = max(2, min(12, self.hyperparams["max_depth"] + random.randint(-2, 2)))
|
| 957 |
+
if random.random() < 0.15:
|
| 958 |
+
self.hyperparams["learning_rate"] *= 10 ** random.uniform(-0.3, 0.3)
|
| 959 |
+
self.hyperparams["learning_rate"] = max(0.001, min(0.5, self.hyperparams["learning_rate"]))
|
| 960 |
+
if random.random() < 0.08:
|
| 961 |
+
self.hyperparams["model_type"] = random.choice(GPU_MODEL_TYPES)
|
| 962 |
+
if random.random() < 0.05:
|
| 963 |
+
# Aligned with HF island weights; venn_abers dominant (brain cycle 82).
|
| 964 |
+
self.hyperparams["calibration"] = random.choices(
|
| 965 |
+
["none", "sigmoid", "venn_abers", "beta", "isotonic", "isotonic_temporal"],
|
| 966 |
+
weights=[16, 12, 25, 20, 14, 13], k=1)[0]
|
| 967 |
+
# Neural net hyperparams
|
| 968 |
+
if random.random() < 0.10:
|
| 969 |
+
self.hyperparams["nn_hidden_dims"] = random.choice([64, 128, 256, 512])
|
| 970 |
+
if random.random() < 0.10:
|
| 971 |
+
self.hyperparams["nn_n_layers"] = max(1, min(6, self.hyperparams.get("nn_n_layers", 2) + random.randint(-1, 1)))
|
| 972 |
+
if random.random() < 0.10:
|
| 973 |
+
self.hyperparams["nn_dropout"] = max(0.0, min(0.7, self.hyperparams.get("nn_dropout", 0.3) + random.uniform(-0.1, 0.1)))
|
| 974 |
+
|
| 975 |
+
|
| 976 |
+
# ═══════════════════════════════════════════════════════════
|
| 977 |
+
# SECTION 4: FITNESS EVALUATION (multi-objective)
|
| 978 |
+
# ═══════════════════════════════════════════════════════════
|
| 979 |
+
|
| 980 |
+
def evaluate_individual(ind, X, y, n_splits=5, use_gpu=False, _eval_counter=[0]):
|
| 981 |
+
"""
|
| 982 |
+
Evaluate one individual via walk-forward backtest.
|
| 983 |
+
Multi-objective: Brier + ROI + Sharpe + Calibration.
|
| 984 |
+
Includes memory management for 16GB RAM with 500 individuals.
|
| 985 |
+
|
| 986 |
+
Post-hoc Platt Scaling (added 2026-03-21):
|
| 987 |
+
Each train fold is split 80/20 into train_proper + calibration_set.
|
| 988 |
+
A LogisticRegression is fitted on (raw_probs_cal, y_cal) and used to
|
| 989 |
+
transform test probabilities → calibrated probabilities before computing
|
| 990 |
+
all downstream metrics (Brier, ROI, ECE). This removes systematic
|
| 991 |
+
over/under-confidence from tree-based models without touching cv=3 inner
|
| 992 |
+
calibration, giving an expected Brier improvement of -0.008 to -0.015.
|
| 993 |
+
"""
|
| 994 |
+
_eval_counter[0] += 1
|
| 995 |
+
if _eval_counter[0] % 10 == 0:
|
| 996 |
+
gc.collect()
|
| 997 |
+
from sklearn.model_selection import TimeSeriesSplit
|
| 998 |
+
from sklearn.metrics import brier_score_loss
|
| 999 |
+
from sklearn.calibration import CalibratedClassifierCV
|
| 1000 |
+
from sklearn.linear_model import LogisticRegression
|
| 1001 |
+
|
| 1002 |
+
selected = ind.selected_indices()
|
| 1003 |
+
if len(selected) < 15 or len(selected) > Individual.MAX_FEATURES:
|
| 1004 |
+
ind.fitness = {"brier": 0.30, "roi": -0.10, "sharpe": -1.0, "calibration": 0.15, "calibration_error": 0.15, "composite": -1.0}
|
| 1005 |
+
return
|
| 1006 |
+
|
| 1007 |
+
X_sub = X[:, selected]
|
| 1008 |
+
X_sub = np.nan_to_num(X_sub, nan=0.0, posinf=1e6, neginf=-1e6)
|
| 1009 |
+
tscv = TimeSeriesSplit(n_splits=n_splits)
|
| 1010 |
+
hp = ind.hyperparams
|
| 1011 |
+
|
| 1012 |
+
model = _build_model(hp, use_gpu)
|
| 1013 |
+
if model is None:
|
| 1014 |
+
ind.fitness["composite"] = -1.0
|
| 1015 |
+
return
|
| 1016 |
+
|
| 1017 |
+
is_icl = hp["model_type"] in ICL_MODEL_TYPES
|
| 1018 |
+
briers, rois, all_probs, all_y = [], [], [], []
|
| 1019 |
+
|
| 1020 |
+
for ti, vi in tscv.split(X_sub):
|
| 1021 |
+
try:
|
| 1022 |
+
# ── ICL models (TabICLv2, TabPFN): no clone via get_params, no calibration wrapper ──
|
| 1023 |
+
if is_icl:
|
| 1024 |
+
m = _build_model(hp, use_gpu)
|
| 1025 |
+
m.fit(X_sub[ti], y[ti])
|
| 1026 |
+
probs = m.predict_proba(X_sub[vi])[:, 1]
|
| 1027 |
+
else:
|
| 1028 |
+
# ── Platt Scaling: split train fold 80/20 → proper + calibration ──
|
| 1029 |
+
cal_split = max(1, int(len(ti) * 0.20))
|
| 1030 |
+
ti_proper = ti[:-cal_split]
|
| 1031 |
+
ti_cal = ti[-cal_split:]
|
| 1032 |
+
|
| 1033 |
+
m = type(model)(**model.get_params())
|
| 1034 |
+
if hp["calibration"] != "none":
|
| 1035 |
+
m = CalibratedClassifierCV(m, method=hp["calibration"], cv=3)
|
| 1036 |
+
|
| 1037 |
+
m.fit(X_sub[ti_proper], y[ti_proper])
|
| 1038 |
+
|
| 1039 |
+
raw_cal = m.predict_proba(X_sub[ti_cal])[:, 1].reshape(-1, 1)
|
| 1040 |
+
y_cal = y[ti_cal]
|
| 1041 |
+
|
| 1042 |
+
platt = LogisticRegression(C=1.0, solver="lbfgs", max_iter=200, random_state=42)
|
| 1043 |
+
platt.fit(raw_cal, y_cal)
|
| 1044 |
+
|
| 1045 |
+
raw_test = m.predict_proba(X_sub[vi])[:, 1].reshape(-1, 1)
|
| 1046 |
+
probs = platt.predict_proba(raw_test)[:, 1]
|
| 1047 |
+
|
| 1048 |
+
briers.append(brier_score_loss(y[vi], probs))
|
| 1049 |
+
rois.append(_simulate_betting(probs, y[vi]))
|
| 1050 |
+
all_probs.extend(probs)
|
| 1051 |
+
all_y.extend(y[vi])
|
| 1052 |
+
except Exception:
|
| 1053 |
+
briers.append(0.28)
|
| 1054 |
+
rois.append(-0.05)
|
| 1055 |
+
|
| 1056 |
+
avg_brier = np.mean(briers)
|
| 1057 |
+
avg_roi = np.mean(rois)
|
| 1058 |
+
sharpe = np.mean(rois) / max(np.std(rois), 0.01) if len(rois) > 1 else 0.0
|
| 1059 |
+
cal_err = _calibration_error(np.array(all_probs), np.array(all_y)) if all_probs else 0.15
|
| 1060 |
+
|
| 1061 |
+
# Multi-objective composite fitness (higher = better)
|
| 1062 |
+
# Feature penalty: penalize bloated individuals (n_features > 80)
|
| 1063 |
+
n_feat = ind.n_features
|
| 1064 |
+
feat_penalty = max(0, (n_feat - 80) / 200) * 0.05 # up to -0.03 for 200 features
|
| 1065 |
+
|
| 1066 |
+
composite = (
|
| 1067 |
+
0.40 * (1 - avg_brier) + # Brier: lower is better
|
| 1068 |
+
0.25 * max(0, avg_roi) + # ROI: higher is better
|
| 1069 |
+
0.20 * max(0, sharpe / 3) + # Sharpe: higher is better
|
| 1070 |
+
0.15 * (1 - cal_err) # Calibration: lower is better
|
| 1071 |
+
- feat_penalty # Parsimony pressure for n_features > 80
|
| 1072 |
+
)
|
| 1073 |
+
|
| 1074 |
+
ind.fitness = {
|
| 1075 |
+
"brier": round(avg_brier, 5),
|
| 1076 |
+
"roi": round(avg_roi, 4),
|
| 1077 |
+
"sharpe": round(sharpe, 4),
|
| 1078 |
+
"calibration": round(cal_err, 4),
|
| 1079 |
+
"calibration_error": round(cal_err, 4), # ECE with 10 bins, on calibrated probs
|
| 1080 |
+
"composite": round(composite, 5),
|
| 1081 |
+
}
|
| 1082 |
+
|
| 1083 |
+
|
| 1084 |
+
def _build_model(hp, use_gpu=False):
|
| 1085 |
+
"""Build ML model from hyperparameters."""
|
| 1086 |
+
mt = hp["model_type"]
|
| 1087 |
+
try:
|
| 1088 |
+
if mt == "xgboost":
|
| 1089 |
+
import xgboost as xgb
|
| 1090 |
+
params = {
|
| 1091 |
+
"n_estimators": hp["n_estimators"],
|
| 1092 |
+
"max_depth": hp["max_depth"],
|
| 1093 |
+
"learning_rate": hp["learning_rate"],
|
| 1094 |
+
"subsample": hp["subsample"],
|
| 1095 |
+
"colsample_bytree": hp["colsample_bytree"],
|
| 1096 |
+
"min_child_weight": hp["min_child_weight"],
|
| 1097 |
+
"reg_alpha": hp["reg_alpha"],
|
| 1098 |
+
"reg_lambda": hp["reg_lambda"],
|
| 1099 |
+
"eval_metric": "logloss",
|
| 1100 |
+
"random_state": 42,
|
| 1101 |
+
"n_jobs": -1,
|
| 1102 |
+
"tree_method": "hist",
|
| 1103 |
+
}
|
| 1104 |
+
if use_gpu:
|
| 1105 |
+
params["device"] = "cuda"
|
| 1106 |
+
return xgb.XGBClassifier(**params)
|
| 1107 |
+
elif mt == "lightgbm":
|
| 1108 |
+
import lightgbm as lgbm
|
| 1109 |
+
return lgbm.LGBMClassifier(
|
| 1110 |
+
n_estimators=hp["n_estimators"],
|
| 1111 |
+
max_depth=hp["max_depth"],
|
| 1112 |
+
learning_rate=hp["learning_rate"],
|
| 1113 |
+
subsample=hp["subsample"],
|
| 1114 |
+
num_leaves=min(2 ** hp["max_depth"] - 1, 127),
|
| 1115 |
+
reg_alpha=hp["reg_alpha"],
|
| 1116 |
+
reg_lambda=hp["reg_lambda"],
|
| 1117 |
+
# DART P012: dropout trees — better probability calibration, reduces reliability error
|
| 1118 |
+
boosting_type="dart", drop_rate=0.1, skip_drop=0.5, uniform_drop=True,
|
| 1119 |
+
verbose=-1, random_state=42, n_jobs=-1,
|
| 1120 |
+
)
|
| 1121 |
+
elif mt == "catboost":
|
| 1122 |
+
from catboost import CatBoostClassifier
|
| 1123 |
+
# CPU speed fix: cap iterations to 60 on CPU (catboost is 3-5x slower than lightgbm)
|
| 1124 |
+
_cat_iters = hp["n_estimators"]
|
| 1125 |
+
if not use_gpu:
|
| 1126 |
+
_cat_iters = min(_cat_iters, 60)
|
| 1127 |
+
_cat_params = dict(
|
| 1128 |
+
iterations=_cat_iters,
|
| 1129 |
+
depth=min(hp["max_depth"], 10),
|
| 1130 |
+
learning_rate=hp["learning_rate"],
|
| 1131 |
+
l2_leaf_reg=hp["reg_lambda"],
|
| 1132 |
+
verbose=0, random_state=42,
|
| 1133 |
+
)
|
| 1134 |
+
if not use_gpu:
|
| 1135 |
+
_cat_params["early_stopping_rounds"] = 15
|
| 1136 |
+
return CatBoostClassifier(**_cat_params)
|
| 1137 |
+
elif mt == "random_forest":
|
| 1138 |
+
from sklearn.ensemble import RandomForestClassifier
|
| 1139 |
+
return RandomForestClassifier(
|
| 1140 |
+
n_estimators=hp["n_estimators"],
|
| 1141 |
+
max_depth=hp["max_depth"],
|
| 1142 |
+
min_samples_leaf=max(1, hp["min_child_weight"]),
|
| 1143 |
+
random_state=42, n_jobs=-1,
|
| 1144 |
+
)
|
| 1145 |
+
elif mt == "extra_trees":
|
| 1146 |
+
from sklearn.ensemble import ExtraTreesClassifier
|
| 1147 |
+
return ExtraTreesClassifier(
|
| 1148 |
+
n_estimators=hp["n_estimators"],
|
| 1149 |
+
max_depth=hp["max_depth"],
|
| 1150 |
+
min_samples_leaf=max(1, hp["min_child_weight"]),
|
| 1151 |
+
random_state=42, n_jobs=-1,
|
| 1152 |
+
)
|
| 1153 |
+
elif mt == "xgboost_brier":
|
| 1154 |
+
import xgboost as xgb
|
| 1155 |
+
def _brier_objective(y_true, y_pred):
|
| 1156 |
+
grad = 2.0 * (y_pred - y_true)
|
| 1157 |
+
hess = np.full_like(grad, 2.0)
|
| 1158 |
+
return grad, hess
|
| 1159 |
+
params = {
|
| 1160 |
+
"n_estimators": hp["n_estimators"],
|
| 1161 |
+
"max_depth": hp["max_depth"],
|
| 1162 |
+
"learning_rate": hp["learning_rate"],
|
| 1163 |
+
"subsample": hp["subsample"],
|
| 1164 |
+
"colsample_bytree": hp["colsample_bytree"],
|
| 1165 |
+
"min_child_weight": hp["min_child_weight"],
|
| 1166 |
+
"reg_alpha": hp["reg_alpha"],
|
| 1167 |
+
"reg_lambda": hp["reg_lambda"],
|
| 1168 |
+
"objective": _brier_objective,
|
| 1169 |
+
"random_state": 42,
|
| 1170 |
+
"n_jobs": -1,
|
| 1171 |
+
"tree_method": "hist",
|
| 1172 |
+
}
|
| 1173 |
+
if use_gpu:
|
| 1174 |
+
params["device"] = "cuda"
|
| 1175 |
+
return xgb.XGBClassifier(**params)
|
| 1176 |
+
elif mt == "tabicl":
|
| 1177 |
+
from tabicl import TabICLClassifier
|
| 1178 |
+
return TabICLClassifier()
|
| 1179 |
+
elif mt == "tabpfn":
|
| 1180 |
+
from tabpfn import TabPFNClassifier
|
| 1181 |
+
return TabPFNClassifier(device="cuda" if use_gpu else "cpu")
|
| 1182 |
+
elif mt == "stacking":
|
| 1183 |
+
from sklearn.ensemble import StackingClassifier, RandomForestClassifier, GradientBoostingClassifier
|
| 1184 |
+
from sklearn.linear_model import LogisticRegression
|
| 1185 |
+
estimators = [
|
| 1186 |
+
("rf", RandomForestClassifier(n_estimators=100, max_depth=hp["max_depth"], random_state=42, n_jobs=-1)),
|
| 1187 |
+
("gb", GradientBoostingClassifier(n_estimators=100, max_depth=min(hp["max_depth"], 6), learning_rate=hp["learning_rate"], random_state=42)),
|
| 1188 |
+
]
|
| 1189 |
+
try:
|
| 1190 |
+
import xgboost as xgb
|
| 1191 |
+
estimators.append(("xgb", xgb.XGBClassifier(n_estimators=100, max_depth=hp["max_depth"], learning_rate=hp["learning_rate"], eval_metric="logloss", random_state=42, n_jobs=-1)))
|
| 1192 |
+
except ImportError:
|
| 1193 |
+
pass
|
| 1194 |
+
return StackingClassifier(estimators=estimators, final_estimator=LogisticRegression(max_iter=500), cv=3, n_jobs=-1)
|
| 1195 |
+
elif mt == "mlp":
|
| 1196 |
+
from sklearn.neural_network import MLPClassifier
|
| 1197 |
+
hidden = tuple([hp.get("nn_hidden_dims", 128)] * hp.get("nn_n_layers", 2))
|
| 1198 |
+
return MLPClassifier(
|
| 1199 |
+
hidden_layer_sizes=hidden,
|
| 1200 |
+
learning_rate_init=hp["learning_rate"],
|
| 1201 |
+
max_iter=hp.get("nn_epochs", 50),
|
| 1202 |
+
alpha=hp["reg_alpha"],
|
| 1203 |
+
random_state=42,
|
| 1204 |
+
)
|
| 1205 |
+
else:
|
| 1206 |
+
# Fallback for unknown types (lstm, transformer, tabnet, etc.) — use GBM
|
| 1207 |
+
from sklearn.ensemble import GradientBoostingClassifier
|
| 1208 |
+
return GradientBoostingClassifier(
|
| 1209 |
+
n_estimators=min(hp["n_estimators"], 200),
|
| 1210 |
+
max_depth=hp["max_depth"],
|
| 1211 |
+
learning_rate=hp["learning_rate"],
|
| 1212 |
+
random_state=42,
|
| 1213 |
+
)
|
| 1214 |
+
except ImportError:
|
| 1215 |
+
from sklearn.ensemble import GradientBoostingClassifier
|
| 1216 |
+
return GradientBoostingClassifier(
|
| 1217 |
+
n_estimators=min(hp["n_estimators"], 200),
|
| 1218 |
+
max_depth=hp["max_depth"],
|
| 1219 |
+
learning_rate=hp["learning_rate"],
|
| 1220 |
+
random_state=42,
|
| 1221 |
+
)
|
| 1222 |
+
return None
|
| 1223 |
+
|
| 1224 |
+
|
| 1225 |
+
def _simulate_betting(probs, actuals, edge=0.05, vig=0.045):
|
| 1226 |
+
"""Simulate flat betting with realistic market odds (including vig).
|
| 1227 |
+
|
| 1228 |
+
Market line estimated as midpoint between our model and 50/50 (conservative).
|
| 1229 |
+
Payout at market decimal odds with vig baked in.
|
| 1230 |
+
This gives a realistic ROI vs the old fair-value (1/prob) approach.
|
| 1231 |
+
"""
|
| 1232 |
+
stake = 10
|
| 1233 |
+
profit = 0
|
| 1234 |
+
n_bets = 0
|
| 1235 |
+
for prob, actual in zip(probs, actuals):
|
| 1236 |
+
# Market prob ~ halfway between our model and 50/50
|
| 1237 |
+
market_prob = 0.5 + (prob - 0.5) * 0.5
|
| 1238 |
+
if prob > 0.5 + edge:
|
| 1239 |
+
# Bet home: market pays at their (less favorable) odds with vig
|
| 1240 |
+
market_decimal = 1.0 / (market_prob * (1 + vig / 2))
|
| 1241 |
+
n_bets += 1
|
| 1242 |
+
if actual == 1:
|
| 1243 |
+
profit += stake * (market_decimal - 1)
|
| 1244 |
+
else:
|
| 1245 |
+
profit -= stake
|
| 1246 |
+
elif prob < 0.5 - edge:
|
| 1247 |
+
# Bet away
|
| 1248 |
+
away_market = 1.0 - market_prob
|
| 1249 |
+
market_decimal = 1.0 / (away_market * (1 + vig / 2))
|
| 1250 |
+
n_bets += 1
|
| 1251 |
+
if actual == 0:
|
| 1252 |
+
profit += stake * (market_decimal - 1)
|
| 1253 |
+
else:
|
| 1254 |
+
profit -= stake
|
| 1255 |
+
return profit / (n_bets * stake) if n_bets > 0 else 0.0
|
| 1256 |
+
|
| 1257 |
+
|
| 1258 |
+
def _calibration_error(probs, actuals, n_bins=10):
|
| 1259 |
+
"""Expected Calibration Error (ECE)."""
|
| 1260 |
+
if len(probs) == 0:
|
| 1261 |
+
return 1.0
|
| 1262 |
+
bins = np.linspace(0, 1, n_bins + 1)
|
| 1263 |
+
ece = 0
|
| 1264 |
+
for i in range(n_bins):
|
| 1265 |
+
mask = (probs >= bins[i]) & (probs < bins[i + 1])
|
| 1266 |
+
if mask.sum() == 0:
|
| 1267 |
+
continue
|
| 1268 |
+
ece += mask.sum() / len(probs) * abs(probs[mask].mean() - actuals[mask].mean())
|
| 1269 |
+
return ece
|
| 1270 |
+
|
| 1271 |
+
|
| 1272 |
+
# ═══════════════════════════════════════════════════════════
|
| 1273 |
+
# SECTION 5: GENETIC EVOLUTION ENGINE
|
| 1274 |
+
# ═══════════════════════════════════════════════════════════
|
| 1275 |
+
|
| 1276 |
+
class GeneticEvolutionEngine:
|
| 1277 |
+
"""
|
| 1278 |
+
REAL genetic evolution engine.
|
| 1279 |
+
Runs continuously, evolving a population of model configs.
|
| 1280 |
+
"""
|
| 1281 |
+
|
| 1282 |
+
def __init__(self, pop_size=500, elite_size=25, mutation_rate=0.15,
|
| 1283 |
+
crossover_rate=0.85, target_features=100, n_splits=3,
|
| 1284 |
+
n_islands=5, migration_interval=10, migrants_per_island=5):
|
| 1285 |
+
self.pop_size = pop_size
|
| 1286 |
+
self.elite_size = elite_size
|
| 1287 |
+
self.base_mutation_rate = mutation_rate
|
| 1288 |
+
self.mutation_rate = mutation_rate
|
| 1289 |
+
self.mut_floor = 0.05
|
| 1290 |
+
self.mut_decay = 0.995
|
| 1291 |
+
self.crossover_rate = crossover_rate
|
| 1292 |
+
self.target_features = target_features
|
| 1293 |
+
self.n_splits = n_splits
|
| 1294 |
+
self.n_islands = n_islands
|
| 1295 |
+
self.island_size = pop_size // n_islands
|
| 1296 |
+
self.migration_interval = migration_interval
|
| 1297 |
+
self.migrants_per_island = migrants_per_island
|
| 1298 |
+
|
| 1299 |
+
self.population = []
|
| 1300 |
+
self.generation = 0
|
| 1301 |
+
self.best_ever = None
|
| 1302 |
+
self.history = []
|
| 1303 |
+
self.stagnation_counter = 0
|
| 1304 |
+
self.use_gpu = False
|
| 1305 |
+
# Hamming diversity tracking
|
| 1306 |
+
self._pop_centroid = None # float list — mean feature mask over population
|
| 1307 |
+
self._hamming_diversity = 1.0 # normalized average pairwise Hamming distance
|
| 1308 |
+
self._no_improve_counter = 0 # gens without best-ever composite improvement
|
| 1309 |
+
|
| 1310 |
+
# Detect GPU
|
| 1311 |
+
try:
|
| 1312 |
+
import xgboost as xgb
|
| 1313 |
+
_test = xgb.XGBClassifier(n_estimators=5, max_depth=3, tree_method="hist", device="cuda")
|
| 1314 |
+
_test.fit(np.random.randn(50, 5), np.random.randint(0, 2, 50))
|
| 1315 |
+
self.use_gpu = True
|
| 1316 |
+
print("[GPU] XGBoost CUDA: ENABLED")
|
| 1317 |
+
except Exception:
|
| 1318 |
+
print("[GPU] XGBoost CUDA: disabled, using CPU")
|
| 1319 |
+
|
| 1320 |
+
def initialize(self, n_features):
|
| 1321 |
+
"""Create initial random population."""
|
| 1322 |
+
self.n_features = n_features
|
| 1323 |
+
self.population = [Individual(n_features, self.target_features) for _ in range(self.pop_size)]
|
| 1324 |
+
print(f"[INIT] Population: {self.pop_size} individuals, {n_features} feature candidates, "
|
| 1325 |
+
f"~{self.target_features} target features")
|
| 1326 |
+
|
| 1327 |
+
def restore_state(self):
|
| 1328 |
+
"""Restore population from saved state (survive restarts)."""
|
| 1329 |
+
state_file = STATE_DIR / "population.json"
|
| 1330 |
+
if not state_file.exists():
|
| 1331 |
+
return False
|
| 1332 |
+
try:
|
| 1333 |
+
state = json.loads(state_file.read_text())
|
| 1334 |
+
self.generation = state["generation"]
|
| 1335 |
+
self.n_features = state["n_features"]
|
| 1336 |
+
self.history = state.get("history", [])
|
| 1337 |
+
self.stagnation_counter = state.get("stagnation_counter", 0)
|
| 1338 |
+
self.mutation_rate = state.get("mutation_rate", self.base_mutation_rate)
|
| 1339 |
+
|
| 1340 |
+
self.population = []
|
| 1341 |
+
for ind_data in state["population"]:
|
| 1342 |
+
ind = Individual.__new__(Individual)
|
| 1343 |
+
ind.features = ind_data["features"]
|
| 1344 |
+
ind.hyperparams = ind_data["hyperparams"]
|
| 1345 |
+
ind.fitness = ind_data["fitness"]
|
| 1346 |
+
ind.generation = ind_data.get("generation", 0)
|
| 1347 |
+
ind.birth_generation = ind_data.get("birth_generation", ind.generation)
|
| 1348 |
+
ind.n_features = sum(ind.features)
|
| 1349 |
+
self.population.append(ind)
|
| 1350 |
+
|
| 1351 |
+
if state.get("best_ever"):
|
| 1352 |
+
be = state["best_ever"]
|
| 1353 |
+
self.best_ever = Individual.__new__(Individual)
|
| 1354 |
+
self.best_ever.features = be["features"]
|
| 1355 |
+
self.best_ever.hyperparams = be["hyperparams"]
|
| 1356 |
+
self.best_ever.fitness = be["fitness"]
|
| 1357 |
+
self.best_ever.generation = be.get("generation", 0)
|
| 1358 |
+
self.best_ever.n_features = sum(self.best_ever.features)
|
| 1359 |
+
|
| 1360 |
+
print(f"[RESTORE] Generation {self.generation}, {len(self.population)} individuals, "
|
| 1361 |
+
f"best Brier={self.best_ever.fitness['brier']:.4f}" if self.best_ever else "")
|
| 1362 |
+
return True
|
| 1363 |
+
except Exception as e:
|
| 1364 |
+
print(f"[RESTORE] Failed: {e}")
|
| 1365 |
+
return False
|
| 1366 |
+
|
| 1367 |
+
def resize_population_features(self, new_n_features):
|
| 1368 |
+
"""Resize feature masks if feature count changed (e.g., new features added)."""
|
| 1369 |
+
old_n = self.n_features
|
| 1370 |
+
if old_n == new_n_features:
|
| 1371 |
+
return
|
| 1372 |
+
delta = new_n_features - old_n
|
| 1373 |
+
print(f"[RESIZE] Feature count changed: {old_n} -> {new_n_features} (delta={delta})")
|
| 1374 |
+
self.n_features = new_n_features
|
| 1375 |
+
for ind in self.population:
|
| 1376 |
+
if len(ind.features) < new_n_features:
|
| 1377 |
+
# Extend with random activation for new features (50% chance each)
|
| 1378 |
+
ind.features.extend([1 if random.random() < 0.3 else 0 for _ in range(new_n_features - len(ind.features))])
|
| 1379 |
+
elif len(ind.features) > new_n_features:
|
| 1380 |
+
ind.features = ind.features[:new_n_features]
|
| 1381 |
+
ind.n_features = sum(ind.features)
|
| 1382 |
+
if self.best_ever:
|
| 1383 |
+
if len(self.best_ever.features) < new_n_features:
|
| 1384 |
+
self.best_ever.features.extend([0] * (new_n_features - len(self.best_ever.features)))
|
| 1385 |
+
elif len(self.best_ever.features) > new_n_features:
|
| 1386 |
+
self.best_ever.features = self.best_ever.features[:new_n_features]
|
| 1387 |
+
self.best_ever.n_features = sum(self.best_ever.features)
|
| 1388 |
+
print(f"[RESIZE] All {len(self.population)} individuals resized")
|
| 1389 |
+
|
| 1390 |
+
def save_state(self):
|
| 1391 |
+
"""Save population state to survive restarts."""
|
| 1392 |
+
state = {
|
| 1393 |
+
"timestamp": datetime.now(timezone.utc).isoformat(),
|
| 1394 |
+
"generation": self.generation,
|
| 1395 |
+
"n_features": self.n_features,
|
| 1396 |
+
"stagnation_counter": self.stagnation_counter,
|
| 1397 |
+
"mutation_rate": self.mutation_rate,
|
| 1398 |
+
"population": [
|
| 1399 |
+
{
|
| 1400 |
+
"features": ind.features,
|
| 1401 |
+
"hyperparams": {k: (float(v) if isinstance(v, (np.floating,)) else v)
|
| 1402 |
+
for k, v in ind.hyperparams.items()},
|
| 1403 |
+
"fitness": ind.fitness,
|
| 1404 |
+
"generation": ind.generation,
|
| 1405 |
+
"birth_generation": getattr(ind, 'birth_generation', ind.generation),
|
| 1406 |
+
}
|
| 1407 |
+
for ind in self.population
|
| 1408 |
+
],
|
| 1409 |
+
"best_ever": {
|
| 1410 |
+
"features": self.best_ever.features,
|
| 1411 |
+
"hyperparams": {k: (float(v) if isinstance(v, (np.floating,)) else v)
|
| 1412 |
+
for k, v in self.best_ever.hyperparams.items()},
|
| 1413 |
+
"fitness": self.best_ever.fitness,
|
| 1414 |
+
"generation": self.best_ever.generation,
|
| 1415 |
+
} if self.best_ever else None,
|
| 1416 |
+
"history": self.history[-200:],
|
| 1417 |
+
}
|
| 1418 |
+
(STATE_DIR / "population.json").write_text(json.dumps(state, default=str))
|
| 1419 |
+
|
| 1420 |
+
# ── Hamming Diversity Utilities ──────────────────────────────────────────
|
| 1421 |
+
|
| 1422 |
+
def _update_pop_centroid(self):
|
| 1423 |
+
"""Compute and cache the population centroid (mean feature mask).
|
| 1424 |
+
|
| 1425 |
+
The centroid[i] is the fraction of individuals that have feature i active.
|
| 1426 |
+
Used by _tournament_select for crowding distance.
|
| 1427 |
+
"""
|
| 1428 |
+
if not self.population:
|
| 1429 |
+
return
|
| 1430 |
+
n = len(self.population[0].features)
|
| 1431 |
+
centroid = [0.0] * n
|
| 1432 |
+
for ind in self.population:
|
| 1433 |
+
for i, v in enumerate(ind.features):
|
| 1434 |
+
centroid[i] += v
|
| 1435 |
+
pop_len = len(self.population)
|
| 1436 |
+
self._pop_centroid = [c / pop_len for c in centroid]
|
| 1437 |
+
|
| 1438 |
+
def _compute_hamming_diversity(self, sample_size=50):
|
| 1439 |
+
"""Compute the normalized average pairwise Hamming distance of the population.
|
| 1440 |
+
|
| 1441 |
+
Exact O(N²) computation is expensive for pop_size=500, so we use a
|
| 1442 |
+
random sample of up to `sample_size` pairs for efficiency.
|
| 1443 |
+
|
| 1444 |
+
Returns a float in [0, 1]. A value of 0 means all feature masks are
|
| 1445 |
+
identical; a value of 1 means every bit differs between every pair.
|
| 1446 |
+
"""
|
| 1447 |
+
pop = self.population
|
| 1448 |
+
if len(pop) < 2:
|
| 1449 |
+
return 1.0
|
| 1450 |
+
n_feat = len(pop[0].features)
|
| 1451 |
+
if n_feat == 0:
|
| 1452 |
+
return 0.0
|
| 1453 |
+
|
| 1454 |
+
# Random sampling: up to sample_size² / 2 pairs
|
| 1455 |
+
indices = list(range(len(pop)))
|
| 1456 |
+
random.shuffle(indices)
|
| 1457 |
+
sample = indices[:sample_size]
|
| 1458 |
+
|
| 1459 |
+
total_dist = 0.0
|
| 1460 |
+
n_pairs = 0
|
| 1461 |
+
for i in range(len(sample)):
|
| 1462 |
+
for j in range(i + 1, len(sample)):
|
| 1463 |
+
f1 = pop[sample[i]].features
|
| 1464 |
+
f2 = pop[sample[j]].features
|
| 1465 |
+
total_dist += sum(a != b for a, b in zip(f1, f2)) / n_feat
|
| 1466 |
+
n_pairs += 1
|
| 1467 |
+
|
| 1468 |
+
return total_dist / n_pairs if n_pairs > 0 else 1.0
|
| 1469 |
+
|
| 1470 |
+
def evolve_one_generation(self, X, y):
|
| 1471 |
+
"""Run one generation of evolution. Returns best individual."""
|
| 1472 |
+
self.generation += 1
|
| 1473 |
+
gen_start = time.time()
|
| 1474 |
+
|
| 1475 |
+
# 1. Evaluate all individuals
|
| 1476 |
+
for i, ind in enumerate(self.population):
|
| 1477 |
+
evaluate_individual(ind, X, y, self.n_splits, self.use_gpu)
|
| 1478 |
+
if (i + 1) % 10 == 0:
|
| 1479 |
+
print(f" Evaluated {i+1}/{len(self.population)}...", end="\r")
|
| 1480 |
+
|
| 1481 |
+
# 2. Sort by composite fitness (higher = better)
|
| 1482 |
+
self.population.sort(key=lambda x: x.fitness["composite"], reverse=True)
|
| 1483 |
+
best = self.population[0]
|
| 1484 |
+
|
| 1485 |
+
# 3. Track best ever
|
| 1486 |
+
prev_best_brier = self.best_ever.fitness["brier"] if self.best_ever else 1.0
|
| 1487 |
+
if self.best_ever is None or best.fitness["composite"] > self.best_ever.fitness["composite"]:
|
| 1488 |
+
self.best_ever = Individual.__new__(Individual)
|
| 1489 |
+
self.best_ever.features = best.features[:]
|
| 1490 |
+
self.best_ever.hyperparams = dict(best.hyperparams)
|
| 1491 |
+
self.best_ever.fitness = dict(best.fitness)
|
| 1492 |
+
self.best_ever.n_features = best.n_features
|
| 1493 |
+
self.best_ever.generation = self.generation
|
| 1494 |
+
|
| 1495 |
+
# 4. Stagnation detection — track BOTH Brier and composite
|
| 1496 |
+
prev_best_composite = self.best_ever.fitness["composite"] if self.best_ever and hasattr(self.best_ever, 'fitness') else 0.0
|
| 1497 |
+
brier_stagnant = abs(best.fitness["brier"] - prev_best_brier) < 0.0005
|
| 1498 |
+
composite_stagnant = abs(best.fitness["composite"] - prev_best_composite) < 0.001
|
| 1499 |
+
if brier_stagnant and composite_stagnant:
|
| 1500 |
+
self.stagnation_counter += 1
|
| 1501 |
+
self._no_improve_counter += 1
|
| 1502 |
+
elif not brier_stagnant:
|
| 1503 |
+
self.stagnation_counter = max(0, self.stagnation_counter - 2) # Partial reset
|
| 1504 |
+
self._no_improve_counter = 0
|
| 1505 |
+
else:
|
| 1506 |
+
self.stagnation_counter = max(0, self.stagnation_counter - 1)
|
| 1507 |
+
self._no_improve_counter = 0
|
| 1508 |
+
|
| 1509 |
+
# 4b. Hamming Diversity Monitor
|
| 1510 |
+
# Compute normalized average pairwise Hamming distance; also refresh centroid
|
| 1511 |
+
# (used by crowding-aware tournament selection below).
|
| 1512 |
+
self._hamming_diversity = self._compute_hamming_diversity(sample_size=50)
|
| 1513 |
+
self._update_pop_centroid()
|
| 1514 |
+
if self._hamming_diversity < 0.15:
|
| 1515 |
+
print(f" [DIVERSITY-LOW] Hamming diversity={self._hamming_diversity:.3f} < 0.15 threshold")
|
| 1516 |
+
|
| 1517 |
+
# 4c. Adaptive Mutation Rate — diversity-driven formula
|
| 1518 |
+
# Base = 0.03; rises smoothly toward 0.10 as diversity falls below 0.25.
|
| 1519 |
+
# Formula: mutation_rate = 0.03 + 0.07 * max(0, 1 - diversity / 0.25)
|
| 1520 |
+
diversity_mutation = 0.03 + 0.07 * max(0.0, 1.0 - self._hamming_diversity / 0.25)
|
| 1521 |
+
# Stagnation boosts applied on top (capped at 0.25)
|
| 1522 |
+
if self.stagnation_counter >= 10:
|
| 1523 |
+
self.mutation_rate = min(0.15, diversity_mutation * 1.8)
|
| 1524 |
+
print(f" [STAGNATION-CRITICAL] {self.stagnation_counter} gens — "
|
| 1525 |
+
f"mutation rate -> {self.mutation_rate:.3f} (diversity={self._hamming_diversity:.3f})")
|
| 1526 |
+
elif self.stagnation_counter >= 7:
|
| 1527 |
+
self.mutation_rate = min(0.15, diversity_mutation * 1.5)
|
| 1528 |
+
print(f" [STAGNATION] {self.stagnation_counter} gens — "
|
| 1529 |
+
f"mutation rate -> {self.mutation_rate:.3f} (diversity={self._hamming_diversity:.3f})")
|
| 1530 |
+
elif self.stagnation_counter >= 3:
|
| 1531 |
+
self.mutation_rate = min(0.12, diversity_mutation * 1.2)
|
| 1532 |
+
else:
|
| 1533 |
+
# Normal regime: formula drives the rate directly
|
| 1534 |
+
self.mutation_rate = diversity_mutation
|
| 1535 |
+
|
| 1536 |
+
# 5. Record history
|
| 1537 |
+
self.history.append({
|
| 1538 |
+
"gen": self.generation,
|
| 1539 |
+
"best_brier": best.fitness["brier"],
|
| 1540 |
+
"best_roi": best.fitness["roi"],
|
| 1541 |
+
"best_sharpe": best.fitness["sharpe"],
|
| 1542 |
+
"best_composite": best.fitness["composite"],
|
| 1543 |
+
"best_calibration_error": best.fitness.get("calibration_error", best.fitness.get("calibration", 1.0)),
|
| 1544 |
+
"n_features": best.n_features,
|
| 1545 |
+
"model_type": best.hyperparams["model_type"],
|
| 1546 |
+
"mutation_rate": round(self.mutation_rate, 4),
|
| 1547 |
+
"avg_composite": round(np.mean([ind.fitness["composite"] for ind in self.population]), 5),
|
| 1548 |
+
"pop_diversity": round(np.std([ind.n_features for ind in self.population]), 1),
|
| 1549 |
+
"hamming_diversity": round(self._hamming_diversity, 4),
|
| 1550 |
+
})
|
| 1551 |
+
|
| 1552 |
+
elapsed = time.time() - gen_start
|
| 1553 |
+
ece_val = best.fitness.get("calibration_error", best.fitness.get("calibration", 1.0))
|
| 1554 |
+
print(f" Gen {self.generation}: Brier={best.fitness['brier']:.4f} "
|
| 1555 |
+
f"ROI={best.fitness['roi']:.1%} Sharpe={best.fitness['sharpe']:.2f} "
|
| 1556 |
+
f"ECE={ece_val:.4f} Features={best.n_features} Model={best.hyperparams['model_type']} "
|
| 1557 |
+
f"Composite={best.fitness['composite']:.4f} "
|
| 1558 |
+
f"Diversity={self._hamming_diversity:.3f} MutRate={self.mutation_rate:.3f} ({elapsed:.0f}s)")
|
| 1559 |
+
|
| 1560 |
+
# 6. Create next generation
|
| 1561 |
+
new_pop = []
|
| 1562 |
+
|
| 1563 |
+
# Elitism — protect top by composite AND top by raw Brier (prevents fossil loss)
|
| 1564 |
+
def _clone_individual(src):
|
| 1565 |
+
clone = Individual.__new__(Individual)
|
| 1566 |
+
clone.features = src.features[:]
|
| 1567 |
+
clone.hyperparams = dict(src.hyperparams)
|
| 1568 |
+
clone.fitness = dict(src.fitness)
|
| 1569 |
+
clone.n_features = src.n_features
|
| 1570 |
+
clone.generation = src.generation
|
| 1571 |
+
clone.birth_generation = getattr(src, 'birth_generation', src.generation)
|
| 1572 |
+
return clone
|
| 1573 |
+
|
| 1574 |
+
# Top elite_size by composite (already sorted)
|
| 1575 |
+
elite_ids = set()
|
| 1576 |
+
for i in range(min(self.elite_size, len(self.population))):
|
| 1577 |
+
new_pop.append(_clone_individual(self.population[i]))
|
| 1578 |
+
elite_ids.add(id(self.population[i]))
|
| 1579 |
+
|
| 1580 |
+
# Also protect top-2 by raw Brier score (lower = better) if not already elite
|
| 1581 |
+
brier_sorted = sorted(self.population, key=lambda x: x.fitness["brier"])
|
| 1582 |
+
for ind in brier_sorted[:2]:
|
| 1583 |
+
if id(ind) not in elite_ids:
|
| 1584 |
+
new_pop.append(_clone_individual(ind))
|
| 1585 |
+
elite_ids.add(id(ind))
|
| 1586 |
+
|
| 1587 |
+
# Aging: remove individuals that have survived > 15 generations without improvement
|
| 1588 |
+
MAX_AGE = 15
|
| 1589 |
+
aged_out = 0
|
| 1590 |
+
for i in range(len(new_pop) - 1, self.elite_size - 1, -1):
|
| 1591 |
+
if i < len(new_pop):
|
| 1592 |
+
age = self.generation - getattr(new_pop[i], 'birth_generation', 0)
|
| 1593 |
+
if age > MAX_AGE and new_pop[i].fitness["composite"] < new_pop[0].fitness["composite"] * 0.95:
|
| 1594 |
+
new_pop.pop(i)
|
| 1595 |
+
aged_out += 1
|
| 1596 |
+
if aged_out > 0:
|
| 1597 |
+
print(f" [AGING] {aged_out} stale individuals removed")
|
| 1598 |
+
|
| 1599 |
+
# Injection: smarter — at stagnation >= 7 inject targeted mutants of best, not just random
|
| 1600 |
+
n_inject = 0
|
| 1601 |
+
if self.stagnation_counter >= 7:
|
| 1602 |
+
n_inject = self.pop_size // 4
|
| 1603 |
+
# Half random, half targeted mutations of best individual
|
| 1604 |
+
n_random = n_inject // 2
|
| 1605 |
+
n_mutant = n_inject - n_random
|
| 1606 |
+
for _ in range(n_random):
|
| 1607 |
+
new_pop.append(Individual(self.n_features, self.target_features))
|
| 1608 |
+
# Targeted mutants: take best, apply heavy mutation
|
| 1609 |
+
for _ in range(n_mutant):
|
| 1610 |
+
mutant = Individual.__new__(Individual)
|
| 1611 |
+
mutant.features = self.population[0].features[:]
|
| 1612 |
+
mutant.hyperparams = dict(self.population[0].hyperparams)
|
| 1613 |
+
mutant.fitness = {"brier": 1.0, "roi": 0.0, "sharpe": 0.0, "calibration": 1.0, "calibration_error": 1.0, "composite": 0.0}
|
| 1614 |
+
mutant.birth_generation = self.generation
|
| 1615 |
+
mutant.n_features = self.population[0].n_features
|
| 1616 |
+
mutant.generation = self.generation
|
| 1617 |
+
mutant.mutate(0.25) # Heavy mutation
|
| 1618 |
+
new_pop.append(mutant)
|
| 1619 |
+
print(f" [INJECTION] {n_random} random + {n_mutant} targeted mutants (stagnation={self.stagnation_counter})")
|
| 1620 |
+
elif self.stagnation_counter >= 3:
|
| 1621 |
+
# Mild injection: 10% fresh individuals
|
| 1622 |
+
n_inject = self.pop_size // 10
|
| 1623 |
+
for _ in range(n_inject):
|
| 1624 |
+
new_pop.append(Individual(self.n_features, self.target_features))
|
| 1625 |
+
print(f" [INJECTION-MILD] {n_inject} fresh individuals (stagnation={self.stagnation_counter})")
|
| 1626 |
+
|
| 1627 |
+
# Diversity Injection: triggered independently when diversity is critically low
|
| 1628 |
+
# (diversity < 0.15) OR when there has been no fitness improvement for 5
|
| 1629 |
+
# consecutive generations — whichever happens first. Elites are always kept.
|
| 1630 |
+
diversity_trigger = (self._hamming_diversity < 0.15) or (self._no_improve_counter >= 5)
|
| 1631 |
+
if diversity_trigger and n_inject == 0:
|
| 1632 |
+
# Inject 20% of population as freshly randomized individuals (elites already in new_pop)
|
| 1633 |
+
n_diversity_inject = max(1, self.pop_size // 5)
|
| 1634 |
+
# Cap to avoid going way over pop_size before the fill loop
|
| 1635 |
+
slots_remaining = max(0, self.pop_size - len(new_pop) - n_diversity_inject)
|
| 1636 |
+
for _ in range(n_diversity_inject):
|
| 1637 |
+
new_pop.append(Individual(self.n_features, self.target_features))
|
| 1638 |
+
trigger_reason = (
|
| 1639 |
+
f"diversity={self._hamming_diversity:.3f}<0.15"
|
| 1640 |
+
if self._hamming_diversity < 0.15
|
| 1641 |
+
else f"no_improve={self._no_improve_counter}>=5"
|
| 1642 |
+
)
|
| 1643 |
+
print(f" [DIVERSITY-INJECT] {n_diversity_inject} fresh individuals injected "
|
| 1644 |
+
f"({trigger_reason}), elites preserved")
|
| 1645 |
+
|
| 1646 |
+
# Fill with crossover + mutation
|
| 1647 |
+
while len(new_pop) < self.pop_size:
|
| 1648 |
+
# Diversity-aware tournament: 80% fitness-based, 20% diversity-based
|
| 1649 |
+
if random.random() < 0.2:
|
| 1650 |
+
p1 = self._diversity_select(7)
|
| 1651 |
+
p2 = self._tournament_select(7)
|
| 1652 |
+
else:
|
| 1653 |
+
p1 = self._tournament_select(7)
|
| 1654 |
+
p2 = self._tournament_select(7)
|
| 1655 |
+
if random.random() < self.crossover_rate:
|
| 1656 |
+
child = Individual.crossover(p1, p2)
|
| 1657 |
+
else:
|
| 1658 |
+
child = Individual.__new__(Individual)
|
| 1659 |
+
child.features = p1.features[:]
|
| 1660 |
+
child.hyperparams = dict(p1.hyperparams)
|
| 1661 |
+
child.fitness = dict(p1.fitness)
|
| 1662 |
+
child.n_features = p1.n_features
|
| 1663 |
+
child.generation = self.generation
|
| 1664 |
+
child.birth_generation = self.generation
|
| 1665 |
+
child.mutate(self.mutation_rate)
|
| 1666 |
+
new_pop.append(child)
|
| 1667 |
+
|
| 1668 |
+
self.population = new_pop[:self.pop_size]
|
| 1669 |
+
return best
|
| 1670 |
+
|
| 1671 |
+
def _tournament_select(self, k=7):
|
| 1672 |
+
"""Tournament selection with crowding.
|
| 1673 |
+
|
| 1674 |
+
Standard tournament selection, but when two candidates have similar
|
| 1675 |
+
composite fitness (within 5%), prefer the one that is more unique —
|
| 1676 |
+
measured by Hamming distance from the population centroid. This
|
| 1677 |
+
implements a lightweight niching pressure that rewards exploration
|
| 1678 |
+
without discarding high-quality individuals.
|
| 1679 |
+
"""
|
| 1680 |
+
contestants = random.sample(self.population, min(k, len(self.population)))
|
| 1681 |
+
best = max(contestants, key=lambda x: x.fitness["composite"])
|
| 1682 |
+
best_fit = best.fitness["composite"]
|
| 1683 |
+
|
| 1684 |
+
# Among contestants within 5% of the best, prefer the most unique one
|
| 1685 |
+
similar = [c for c in contestants if best_fit > 0 and
|
| 1686 |
+
abs(c.fitness["composite"] - best_fit) / max(abs(best_fit), 1e-9) < 0.05]
|
| 1687 |
+
if len(similar) > 1 and hasattr(self, '_pop_centroid') and self._pop_centroid is not None:
|
| 1688 |
+
centroid = self._pop_centroid
|
| 1689 |
+
def _dist_from_centroid(ind):
|
| 1690 |
+
f = ind.features
|
| 1691 |
+
n = len(f)
|
| 1692 |
+
if n == 0 or len(centroid) != n:
|
| 1693 |
+
return 0.0
|
| 1694 |
+
return sum(abs(f[i] - centroid[i]) for i in range(n)) / n
|
| 1695 |
+
best = max(similar, key=_dist_from_centroid)
|
| 1696 |
+
|
| 1697 |
+
return best
|
| 1698 |
+
|
| 1699 |
+
def _diversity_select(self, k=7):
|
| 1700 |
+
"""Diversity-preserving selection: pick the most unique individual from k random."""
|
| 1701 |
+
contestants = random.sample(self.population, min(k, len(self.population)))
|
| 1702 |
+
if not self.population:
|
| 1703 |
+
return contestants[0]
|
| 1704 |
+
# Measure uniqueness: how different is this individual's feature set from the elite?
|
| 1705 |
+
elite_features = set()
|
| 1706 |
+
for i, ind in enumerate(self.population[:self.elite_size]):
|
| 1707 |
+
elite_features.update(ind.selected_indices())
|
| 1708 |
+
best_diversity = -1
|
| 1709 |
+
best_ind = contestants[0]
|
| 1710 |
+
for c in contestants:
|
| 1711 |
+
c_features = set(c.selected_indices())
|
| 1712 |
+
if not c_features:
|
| 1713 |
+
continue
|
| 1714 |
+
overlap = len(c_features & elite_features) / max(len(c_features), 1)
|
| 1715 |
+
diversity = 1.0 - overlap
|
| 1716 |
+
# Weight by fitness to avoid picking terrible individuals
|
| 1717 |
+
score = diversity * 0.6 + max(0, c.fitness["composite"]) * 0.4
|
| 1718 |
+
if score > best_diversity:
|
| 1719 |
+
best_diversity = score
|
| 1720 |
+
best_ind = c
|
| 1721 |
+
return best_ind
|
| 1722 |
+
|
| 1723 |
+
def save_cycle_results(self, feature_names):
|
| 1724 |
+
"""Save results after a cycle of generations."""
|
| 1725 |
+
if not self.best_ever:
|
| 1726 |
+
return
|
| 1727 |
+
|
| 1728 |
+
selected_names = [feature_names[i] for i in self.best_ever.selected_indices()
|
| 1729 |
+
if i < len(feature_names)]
|
| 1730 |
+
|
| 1731 |
+
results = {
|
| 1732 |
+
"timestamp": datetime.now(timezone.utc).isoformat(),
|
| 1733 |
+
"generation": self.generation,
|
| 1734 |
+
"population_size": self.pop_size,
|
| 1735 |
+
"feature_candidates": self.n_features,
|
| 1736 |
+
"mutation_rate": round(self.mutation_rate, 4),
|
| 1737 |
+
"stagnation_counter": self.stagnation_counter,
|
| 1738 |
+
"gpu": self.use_gpu,
|
| 1739 |
+
"best": {
|
| 1740 |
+
"brier": self.best_ever.fitness["brier"],
|
| 1741 |
+
"roi": self.best_ever.fitness["roi"],
|
| 1742 |
+
"sharpe": self.best_ever.fitness["sharpe"],
|
| 1743 |
+
"calibration": self.best_ever.fitness["calibration"],
|
| 1744 |
+
"calibration_error": self.best_ever.fitness.get("calibration_error", self.best_ever.fitness["calibration"]),
|
| 1745 |
+
"composite": self.best_ever.fitness["composite"],
|
| 1746 |
+
"n_features": self.best_ever.n_features,
|
| 1747 |
+
"model_type": self.best_ever.hyperparams["model_type"],
|
| 1748 |
+
"hyperparams": {k: (float(v) if isinstance(v, (np.floating, np.integer)) else v)
|
| 1749 |
+
for k, v in self.best_ever.hyperparams.items()},
|
| 1750 |
+
"selected_features": selected_names[:50],
|
| 1751 |
+
},
|
| 1752 |
+
"top5": [ind.to_dict() for ind in sorted(
|
| 1753 |
+
self.population, key=lambda x: x.fitness["composite"], reverse=True
|
| 1754 |
+
)[:5]],
|
| 1755 |
+
"history_last20": self.history[-20:],
|
| 1756 |
+
}
|
| 1757 |
+
|
| 1758 |
+
# Save timestamped + latest
|
| 1759 |
+
ts = datetime.now().strftime("%Y%m%d-%H%M")
|
| 1760 |
+
(RESULTS_DIR / f"evolution-{ts}.json").write_text(json.dumps(results, indent=2, default=str))
|
| 1761 |
+
(RESULTS_DIR / "evolution-latest.json").write_text(json.dumps(results, indent=2, default=str))
|
| 1762 |
+
return results
|
| 1763 |
+
|
| 1764 |
+
|
| 1765 |
+
# ═══════════════════════════════════════════════════════════
|
| 1766 |
+
# SECTION 6: VM CALLBACK
|
| 1767 |
+
# ═══════════════════════════════════════════════════════════
|
| 1768 |
+
|
| 1769 |
+
def callback_to_vm(results):
|
| 1770 |
+
"""POST results to VM data server (best-effort)."""
|
| 1771 |
+
import urllib.request
|
| 1772 |
+
try:
|
| 1773 |
+
url = f"{VM_CALLBACK_URL}/callback/evolution"
|
| 1774 |
+
body = json.dumps(results, default=str).encode()
|
| 1775 |
+
req = urllib.request.Request(url, data=body, headers={"Content-Type": "application/json"})
|
| 1776 |
+
resp = urllib.request.urlopen(req, timeout=10)
|
| 1777 |
+
print(f" [CALLBACK] VM notified: {resp.status}")
|
| 1778 |
+
except Exception as e:
|
| 1779 |
+
# Best-effort, don't block on failure
|
| 1780 |
+
print(f" [CALLBACK] VM unreachable: {e}")
|
| 1781 |
+
|
| 1782 |
+
# Also try to write to shared mon-ipad data if accessible
|
| 1783 |
+
try:
|
| 1784 |
+
shared = Path("/home/termius/mon-ipad/data/nba-agent/evolution-latest.json")
|
| 1785 |
+
if shared.parent.exists():
|
| 1786 |
+
shared.write_text(json.dumps(results, indent=2, default=str))
|
| 1787 |
+
print(f" [CALLBACK] Wrote to mon-ipad")
|
| 1788 |
+
except Exception:
|
| 1789 |
+
pass
|
| 1790 |
+
|
| 1791 |
+
|
| 1792 |
+
# ═══════════════════════════════════════════════════════════
|
| 1793 |
+
# SECTION 7: MAIN LOOP (continuous 24/7)
|
| 1794 |
+
# ═══════════════════════════════════════════════════════════
|
| 1795 |
+
|
| 1796 |
+
def run_continuous(generations_per_cycle=10, total_cycles=None, pop_size=500,
|
| 1797 |
+
target_features=100, n_splits=5, cool_down=30):
|
| 1798 |
+
"""
|
| 1799 |
+
Main entry point — runs genetic evolution CONTINUOUSLY.
|
| 1800 |
+
|
| 1801 |
+
Args:
|
| 1802 |
+
generations_per_cycle: Generations per cycle before saving/callback
|
| 1803 |
+
total_cycles: None = infinite (24/7 mode)
|
| 1804 |
+
pop_size: Population size
|
| 1805 |
+
target_features: Target number of features per individual
|
| 1806 |
+
n_splits: Walk-forward backtest splits
|
| 1807 |
+
cool_down: Seconds between cycles
|
| 1808 |
+
"""
|
| 1809 |
+
print("=" * 70)
|
| 1810 |
+
print(" NBA QUANT AI — REAL GENETIC EVOLUTION LOOP v3")
|
| 1811 |
+
print(f" Started: {datetime.now(timezone.utc).isoformat()}")
|
| 1812 |
+
print(f" Pop: {pop_size} | Target features: {target_features}")
|
| 1813 |
+
print(f" Gens/cycle: {generations_per_cycle} | Cycles: {'INFINITE' if total_cycles is None else total_cycles}")
|
| 1814 |
+
print("=" * 70)
|
| 1815 |
+
|
| 1816 |
+
# 1. Pull data
|
| 1817 |
+
print("\n[PHASE 1] Loading data...")
|
| 1818 |
+
pull_seasons()
|
| 1819 |
+
games = load_all_games()
|
| 1820 |
+
print(f" {len(games)} games loaded")
|
| 1821 |
+
if len(games) < 500:
|
| 1822 |
+
print(" ERROR: Not enough games!")
|
| 1823 |
+
return
|
| 1824 |
+
|
| 1825 |
+
# 2. Build features
|
| 1826 |
+
print("\n[PHASE 2] Building features...")
|
| 1827 |
+
X, y, feature_names = build_features(games)
|
| 1828 |
+
print(f" Feature matrix: {X.shape} ({len(feature_names)} features)")
|
| 1829 |
+
|
| 1830 |
+
# 3. Initialize engine
|
| 1831 |
+
print("\n[PHASE 3] Initializing engine...")
|
| 1832 |
+
engine = GeneticEvolutionEngine(
|
| 1833 |
+
pop_size=pop_size, elite_size=max(5, pop_size // 20), mutation_rate=0.15,
|
| 1834 |
+
crossover_rate=0.85, target_features=target_features, n_splits=n_splits,
|
| 1835 |
+
n_islands=5, migration_interval=10, migrants_per_island=5,
|
| 1836 |
+
)
|
| 1837 |
+
|
| 1838 |
+
# Try to restore previous state
|
| 1839 |
+
if not engine.restore_state():
|
| 1840 |
+
engine.initialize(X.shape[1])
|
| 1841 |
+
else:
|
| 1842 |
+
# Resize population if feature count changed (new features added)
|
| 1843 |
+
engine.resize_population_features(X.shape[1])
|
| 1844 |
+
|
| 1845 |
+
# ── Supabase Run Logger + Auto-Cut ──
|
| 1846 |
+
run_logger = None
|
| 1847 |
+
if _HAS_LOGGER:
|
| 1848 |
+
try:
|
| 1849 |
+
run_logger = RunLogger(local_dir=str(RESULTS_DIR / "run-logs"))
|
| 1850 |
+
print("[RUN-LOGGER] Supabase logging + auto-cut ACTIVE")
|
| 1851 |
+
except Exception as e:
|
| 1852 |
+
print(f"[RUN-LOGGER] Init failed: {e}")
|
| 1853 |
+
|
| 1854 |
+
# 4. CONTINUOUS EVOLUTION LOOP
|
| 1855 |
+
cycle = 0
|
| 1856 |
+
while True:
|
| 1857 |
+
cycle += 1
|
| 1858 |
+
if total_cycles is not None and cycle > total_cycles:
|
| 1859 |
+
break
|
| 1860 |
+
|
| 1861 |
+
cycle_start = time.time()
|
| 1862 |
+
print(f"\n{'='*60}")
|
| 1863 |
+
print(f" CYCLE {cycle} — Starting {generations_per_cycle} generations")
|
| 1864 |
+
print(f" Time: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')}")
|
| 1865 |
+
print(f"{'='*60}")
|
| 1866 |
+
|
| 1867 |
+
for gen in range(generations_per_cycle):
|
| 1868 |
+
try:
|
| 1869 |
+
gen_start = time.time()
|
| 1870 |
+
best = engine.evolve_one_generation(X, y)
|
| 1871 |
+
|
| 1872 |
+
# ── Log generation + auto-cut ──
|
| 1873 |
+
if run_logger and best:
|
| 1874 |
+
try:
|
| 1875 |
+
pop_div = float(np.std([ind.n_features for ind in engine.population]))
|
| 1876 |
+
avg_comp = float(np.mean([ind.fitness["composite"] for ind in engine.population]))
|
| 1877 |
+
run_logger.log_generation(
|
| 1878 |
+
cycle=cycle, generation=engine.generation,
|
| 1879 |
+
best={"brier": best.fitness["brier"], "roi": best.fitness["roi"],
|
| 1880 |
+
"sharpe": best.fitness["sharpe"], "composite": best.fitness["composite"],
|
| 1881 |
+
"n_features": best.n_features, "model_type": best.hyperparams["model_type"]},
|
| 1882 |
+
mutation_rate=engine.mutation_rate, avg_composite=avg_comp,
|
| 1883 |
+
pop_diversity=pop_div, duration_s=time.time() - gen_start)
|
| 1884 |
+
|
| 1885 |
+
# Auto-cut check
|
| 1886 |
+
cut_actions = run_logger.check_auto_cut(best.fitness, {
|
| 1887 |
+
"mutation_rate": engine.mutation_rate,
|
| 1888 |
+
"stagnation": engine.stagnation_counter,
|
| 1889 |
+
"pop_size": engine.pop_size,
|
| 1890 |
+
"pop_diversity": pop_div,
|
| 1891 |
+
})
|
| 1892 |
+
for action in cut_actions:
|
| 1893 |
+
atype = action["type"]
|
| 1894 |
+
params = action.get("params", {})
|
| 1895 |
+
if atype == "config" and "mutation_rate" in params:
|
| 1896 |
+
engine.mutation_rate = params["mutation_rate"]
|
| 1897 |
+
elif atype == "emergency_diversify":
|
| 1898 |
+
n_new = engine.pop_size // 3
|
| 1899 |
+
engine.population = sorted(engine.population, key=lambda x: x.fitness["composite"], reverse=True)[:engine.pop_size - n_new]
|
| 1900 |
+
for _ in range(n_new):
|
| 1901 |
+
engine.population.append(Individual(engine.n_features, engine.target_features))
|
| 1902 |
+
print(f" [AUTO-CUT] Diversified: {n_new} fresh individuals")
|
| 1903 |
+
elif atype == "full_reset":
|
| 1904 |
+
engine.population = sorted(engine.population, key=lambda x: x.fitness["composite"], reverse=True)[:engine.elite_size]
|
| 1905 |
+
while len(engine.population) < engine.pop_size:
|
| 1906 |
+
engine.population.append(Individual(engine.n_features, engine.target_features))
|
| 1907 |
+
engine.stagnation_counter = 0
|
| 1908 |
+
print(f" [AUTO-CUT] FULL RESET executed")
|
| 1909 |
+
except Exception as e:
|
| 1910 |
+
print(f" [RUN-LOGGER] Error: {e}")
|
| 1911 |
+
except Exception as e:
|
| 1912 |
+
print(f" [ERROR] Generation failed: {e}")
|
| 1913 |
+
traceback.print_exc()
|
| 1914 |
+
continue
|
| 1915 |
+
|
| 1916 |
+
# Save state (survives restarts)
|
| 1917 |
+
engine.save_state()
|
| 1918 |
+
|
| 1919 |
+
# Save results
|
| 1920 |
+
results = engine.save_cycle_results(feature_names)
|
| 1921 |
+
|
| 1922 |
+
cycle_elapsed = time.time() - cycle_start
|
| 1923 |
+
print(f"\n Cycle {cycle} complete in {cycle_elapsed:.0f}s")
|
| 1924 |
+
|
| 1925 |
+
if engine.best_ever:
|
| 1926 |
+
print(f" BEST EVER: Brier={engine.best_ever.fitness['brier']:.4f} "
|
| 1927 |
+
f"ROI={engine.best_ever.fitness['roi']:.1%} "
|
| 1928 |
+
f"Features={engine.best_ever.n_features}")
|
| 1929 |
+
|
| 1930 |
+
# ── Log cycle to Supabase ──
|
| 1931 |
+
if run_logger and results and engine.best_ever:
|
| 1932 |
+
try:
|
| 1933 |
+
pop_div = float(np.std([ind.n_features for ind in engine.population]))
|
| 1934 |
+
avg_comp = float(np.mean([ind.fitness["composite"] for ind in engine.population]))
|
| 1935 |
+
run_logger.log_cycle(
|
| 1936 |
+
cycle=cycle, generation=engine.generation,
|
| 1937 |
+
best=engine.best_ever.fitness | {"n_features": engine.best_ever.n_features,
|
| 1938 |
+
"model_type": engine.best_ever.hyperparams["model_type"]},
|
| 1939 |
+
pop_size=engine.pop_size, mutation_rate=engine.mutation_rate,
|
| 1940 |
+
crossover_rate=engine.crossover_rate, stagnation=engine.stagnation_counter,
|
| 1941 |
+
games=len(games), feature_candidates=X.shape[1],
|
| 1942 |
+
cycle_duration_s=cycle_elapsed, avg_composite=avg_comp, pop_diversity=pop_div,
|
| 1943 |
+
top5=results.get("top5"), selected_features=results.get("best", {}).get("selected_features"))
|
| 1944 |
+
print(f" [RUN-LOGGER] Cycle {cycle} logged to Supabase")
|
| 1945 |
+
except Exception as e:
|
| 1946 |
+
print(f" [RUN-LOGGER] Cycle log error: {e}")
|
| 1947 |
+
|
| 1948 |
+
# Callback to VM
|
| 1949 |
+
if results:
|
| 1950 |
+
callback_to_vm(results)
|
| 1951 |
+
|
| 1952 |
+
# Refresh data periodically (every 10 cycles)
|
| 1953 |
+
if cycle % 10 == 0:
|
| 1954 |
+
print("\n [REFRESH] Pulling latest game data...")
|
| 1955 |
+
try:
|
| 1956 |
+
pull_seasons()
|
| 1957 |
+
new_games = load_all_games()
|
| 1958 |
+
if len(new_games) > len(games):
|
| 1959 |
+
games = new_games
|
| 1960 |
+
X, y, feature_names = build_features(games)
|
| 1961 |
+
print(f" [REFRESH] Updated: {X.shape}")
|
| 1962 |
+
except Exception as e:
|
| 1963 |
+
print(f" [REFRESH] Failed: {e}")
|
| 1964 |
+
|
| 1965 |
+
if total_cycles is None:
|
| 1966 |
+
print(f"\n Cooling down {cool_down}s before next cycle...")
|
| 1967 |
+
time.sleep(cool_down)
|
| 1968 |
+
|
| 1969 |
+
print("\n" + "=" * 70)
|
| 1970 |
+
print(" EVOLUTION COMPLETE")
|
| 1971 |
+
if engine.best_ever:
|
| 1972 |
+
print(f" Final best: Brier={engine.best_ever.fitness['brier']:.4f} "
|
| 1973 |
+
f"ROI={engine.best_ever.fitness['roi']:.1%}")
|
| 1974 |
+
print("=" * 70)
|
| 1975 |
+
|
| 1976 |
+
|
| 1977 |
+
# ═══════════════════════════════════════════════════════════
|
| 1978 |
+
# CLI ENTRY POINT
|
| 1979 |
+
# ═══════════════════════════════════════════════════════════
|
| 1980 |
+
|
| 1981 |
+
if __name__ == "__main__":
|
| 1982 |
+
import argparse
|
| 1983 |
+
parser = argparse.ArgumentParser(description="NBA Quant Genetic Evolution Loop v3")
|
| 1984 |
+
parser.add_argument("--continuous", action="store_true", help="Run 24/7 (no cycle limit)")
|
| 1985 |
+
parser.add_argument("--generations", type=int, default=10, help="Generations per cycle (default: 10)")
|
| 1986 |
+
parser.add_argument("--cycles", type=int, default=None, help="Number of cycles (default: infinite)")
|
| 1987 |
+
parser.add_argument("--pop-size", type=int, default=500, help="Population size (default: 500)")
|
| 1988 |
+
parser.add_argument("--target-features", type=int, default=100, help="Target features (default: 100)")
|
| 1989 |
+
parser.add_argument("--splits", type=int, default=5, help="Walk-forward splits (default: 5)")
|
| 1990 |
+
parser.add_argument("--cooldown", type=int, default=30, help="Seconds between cycles (default: 30)")
|
| 1991 |
+
args = parser.parse_args()
|
| 1992 |
+
|
| 1993 |
+
cycles = None if args.continuous else (args.cycles or 1)
|
| 1994 |
+
|
| 1995 |
+
run_continuous(
|
| 1996 |
+
generations_per_cycle=args.generations,
|
| 1997 |
+
total_cycles=cycles,
|
| 1998 |
+
pop_size=args.pop_size,
|
| 1999 |
+
target_features=args.target_features,
|
| 2000 |
+
n_splits=args.splits,
|
| 2001 |
+
cool_down=args.cooldown,
|
| 2002 |
+
)
|
evolution/run_logger.py
ADDED
|
@@ -0,0 +1,425 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Run Logger & Auto-Cut — Hedge Fund Grade Evolution Monitoring
|
| 4 |
+
================================================================
|
| 5 |
+
Logs EVERY generation, cycle, and eval to Supabase.
|
| 6 |
+
Auto-cuts evolution when regression detected or stagnation exceeds threshold.
|
| 7 |
+
|
| 8 |
+
Tables (auto-created):
|
| 9 |
+
- nba_evolution_runs : one row per cycle (summary)
|
| 10 |
+
- nba_evolution_gens : one row per generation (detailed)
|
| 11 |
+
- nba_evolution_evals : one row per individual evaluation
|
| 12 |
+
- nba_evolution_cuts : log of auto-cut events
|
| 13 |
+
|
| 14 |
+
Auto-Cut Rules:
|
| 15 |
+
1. REGRESSION CUT: If best Brier increases by > 0.005 for 3 consecutive gens → rollback
|
| 16 |
+
2. STAGNATION CUT: If no improvement for 20 gens → emergency diversify + log
|
| 17 |
+
3. ROI CUT: If ROI drops below -15% → pause betting, continue evolving
|
| 18 |
+
4. DIVERSITY CUT: If population diversity < 0.05 → inject fresh individuals
|
| 19 |
+
5. FEATURE CUT: If selected features < 40 → expand target_features
|
| 20 |
+
|
| 21 |
+
Designed for real-time monitoring via Supabase dashboard or Telegram.
|
| 22 |
+
"""
|
| 23 |
+
|
| 24 |
+
import os
|
| 25 |
+
import json
|
| 26 |
+
import time
|
| 27 |
+
from datetime import datetime, timezone
|
| 28 |
+
from pathlib import Path
|
| 29 |
+
from typing import Dict, List, Optional
|
| 30 |
+
|
| 31 |
+
# ── Supabase connection ──
|
| 32 |
+
_SUPABASE_URL = os.environ.get("SUPABASE_URL", "")
|
| 33 |
+
_DATABASE_URL = os.environ.get("DATABASE_URL", "")
|
| 34 |
+
_SUPABASE_KEY = os.environ.get("SUPABASE_API_KEY", os.environ.get("SUPABASE_ANON_KEY", ""))
|
| 35 |
+
|
| 36 |
+
_pg_pool = None
|
| 37 |
+
|
| 38 |
+
def _get_pg():
|
| 39 |
+
"""Lazy PostgreSQL connection pool (Supabase = PostgreSQL)."""
|
| 40 |
+
global _pg_pool
|
| 41 |
+
if _pg_pool is not None:
|
| 42 |
+
return _pg_pool
|
| 43 |
+
# Read at call time, not import time (HF Spaces set env late)
|
| 44 |
+
db_url = os.environ.get("DATABASE_URL", "") or _DATABASE_URL
|
| 45 |
+
if not db_url:
|
| 46 |
+
print(f"[RUN-LOGGER] DATABASE_URL not set — Supabase logging disabled")
|
| 47 |
+
return None
|
| 48 |
+
try:
|
| 49 |
+
import psycopg2
|
| 50 |
+
from psycopg2 import pool as pg_pool
|
| 51 |
+
_pg_pool = pg_pool.SimpleConnectionPool(1, 3, db_url, options="-c search_path=public")
|
| 52 |
+
# Test the connection
|
| 53 |
+
conn = _pg_pool.getconn()
|
| 54 |
+
with conn.cursor() as cur:
|
| 55 |
+
cur.execute("SELECT 1")
|
| 56 |
+
_pg_pool.putconn(conn)
|
| 57 |
+
print(f"[RUN-LOGGER] PostgreSQL connected to Supabase OK")
|
| 58 |
+
return _pg_pool
|
| 59 |
+
except Exception as e:
|
| 60 |
+
print(f"[RUN-LOGGER] PostgreSQL connection failed: {e}")
|
| 61 |
+
_pg_pool = None
|
| 62 |
+
return None
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def _exec_sql(sql, params=None):
|
| 66 |
+
"""Execute SQL on Supabase PostgreSQL. Best-effort, never crashes."""
|
| 67 |
+
pool = _get_pg()
|
| 68 |
+
if not pool:
|
| 69 |
+
return None
|
| 70 |
+
conn = None
|
| 71 |
+
try:
|
| 72 |
+
conn = pool.getconn()
|
| 73 |
+
with conn.cursor() as cur:
|
| 74 |
+
cur.execute(sql, params)
|
| 75 |
+
conn.commit()
|
| 76 |
+
try:
|
| 77 |
+
return cur.fetchall()
|
| 78 |
+
except Exception:
|
| 79 |
+
return True # INSERT/UPDATE succeeded but no rows to fetch
|
| 80 |
+
except Exception as e:
|
| 81 |
+
if conn:
|
| 82 |
+
try:
|
| 83 |
+
conn.rollback()
|
| 84 |
+
except Exception:
|
| 85 |
+
pass
|
| 86 |
+
print(f"[RUN-LOGGER] SQL error: {e}", flush=True)
|
| 87 |
+
return None
|
| 88 |
+
finally:
|
| 89 |
+
if conn and pool:
|
| 90 |
+
try:
|
| 91 |
+
pool.putconn(conn)
|
| 92 |
+
except Exception:
|
| 93 |
+
pass
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def _ensure_tables():
|
| 97 |
+
"""Create logging tables if they don't exist."""
|
| 98 |
+
sqls = [
|
| 99 |
+
"""CREATE TABLE IF NOT EXISTS public.nba_evolution_runs (
|
| 100 |
+
id SERIAL PRIMARY KEY,
|
| 101 |
+
ts TIMESTAMPTZ DEFAULT NOW(),
|
| 102 |
+
cycle INT,
|
| 103 |
+
generation INT,
|
| 104 |
+
best_brier FLOAT,
|
| 105 |
+
best_roi FLOAT,
|
| 106 |
+
best_sharpe FLOAT,
|
| 107 |
+
best_calibration FLOAT,
|
| 108 |
+
best_composite FLOAT,
|
| 109 |
+
best_features INT,
|
| 110 |
+
best_model_type TEXT,
|
| 111 |
+
pop_size INT,
|
| 112 |
+
mutation_rate FLOAT,
|
| 113 |
+
crossover_rate FLOAT,
|
| 114 |
+
stagnation INT,
|
| 115 |
+
games INT,
|
| 116 |
+
feature_candidates INT,
|
| 117 |
+
cycle_duration_s FLOAT,
|
| 118 |
+
avg_composite FLOAT,
|
| 119 |
+
pop_diversity FLOAT,
|
| 120 |
+
top5 JSONB,
|
| 121 |
+
selected_features JSONB
|
| 122 |
+
)""",
|
| 123 |
+
"""CREATE TABLE IF NOT EXISTS public.nba_evolution_gens (
|
| 124 |
+
id SERIAL PRIMARY KEY,
|
| 125 |
+
ts TIMESTAMPTZ DEFAULT NOW(),
|
| 126 |
+
cycle INT,
|
| 127 |
+
generation INT,
|
| 128 |
+
best_brier FLOAT,
|
| 129 |
+
best_roi FLOAT,
|
| 130 |
+
best_sharpe FLOAT,
|
| 131 |
+
best_composite FLOAT,
|
| 132 |
+
n_features INT,
|
| 133 |
+
model_type TEXT,
|
| 134 |
+
mutation_rate FLOAT,
|
| 135 |
+
avg_composite FLOAT,
|
| 136 |
+
pop_diversity FLOAT,
|
| 137 |
+
gen_duration_s FLOAT,
|
| 138 |
+
improved BOOLEAN DEFAULT FALSE
|
| 139 |
+
)""",
|
| 140 |
+
"""CREATE TABLE IF NOT EXISTS public.nba_evolution_cuts (
|
| 141 |
+
id SERIAL PRIMARY KEY,
|
| 142 |
+
ts TIMESTAMPTZ DEFAULT NOW(),
|
| 143 |
+
cut_type TEXT,
|
| 144 |
+
reason TEXT,
|
| 145 |
+
brier_before FLOAT,
|
| 146 |
+
brier_after FLOAT,
|
| 147 |
+
action_taken TEXT,
|
| 148 |
+
params_applied JSONB
|
| 149 |
+
)""",
|
| 150 |
+
"""CREATE TABLE IF NOT EXISTS public.nba_evolution_evals (
|
| 151 |
+
id SERIAL PRIMARY KEY,
|
| 152 |
+
ts TIMESTAMPTZ DEFAULT NOW(),
|
| 153 |
+
generation INT,
|
| 154 |
+
individual_rank INT,
|
| 155 |
+
brier FLOAT,
|
| 156 |
+
roi FLOAT,
|
| 157 |
+
sharpe FLOAT,
|
| 158 |
+
composite FLOAT,
|
| 159 |
+
n_features INT,
|
| 160 |
+
model_type TEXT
|
| 161 |
+
)""",
|
| 162 |
+
]
|
| 163 |
+
for sql in sqls:
|
| 164 |
+
_exec_sql(sql)
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
# ── Auto-initialize tables on import ──
|
| 168 |
+
_tables_ready = False
|
| 169 |
+
|
| 170 |
+
|
| 171 |
+
class RunLogger:
|
| 172 |
+
"""Logs evolution runs to Supabase + local files. Never crashes the main loop."""
|
| 173 |
+
|
| 174 |
+
def __init__(self, local_dir=None):
|
| 175 |
+
global _tables_ready
|
| 176 |
+
self.local_dir = Path(local_dir or "/data/run-logs")
|
| 177 |
+
self.local_dir.mkdir(parents=True, exist_ok=True)
|
| 178 |
+
|
| 179 |
+
# Auto-cut state
|
| 180 |
+
self.brier_history = [] # last N best Brier values
|
| 181 |
+
self.regression_count = 0 # consecutive regressions
|
| 182 |
+
self.stagnation_count = 0
|
| 183 |
+
self.last_best_brier = 1.0
|
| 184 |
+
self.last_best_composite = 0.0
|
| 185 |
+
self.cuts_applied = 0
|
| 186 |
+
|
| 187 |
+
# Ensure Supabase tables exist
|
| 188 |
+
if not _tables_ready:
|
| 189 |
+
_ensure_tables()
|
| 190 |
+
_tables_ready = True
|
| 191 |
+
print("[RUN-LOGGER] Supabase tables ready")
|
| 192 |
+
|
| 193 |
+
# ══════════════════════════════════════════
|
| 194 |
+
# LOG — Record events
|
| 195 |
+
# ══════════════════════════════════════════
|
| 196 |
+
|
| 197 |
+
def log_generation(self, cycle, generation, best, mutation_rate, avg_composite, pop_diversity, duration_s):
|
| 198 |
+
"""Log one generation result."""
|
| 199 |
+
improved = best["brier"] < self.last_best_brier - 0.0001
|
| 200 |
+
|
| 201 |
+
# Supabase
|
| 202 |
+
_exec_sql("""INSERT INTO public.nba_evolution_gens
|
| 203 |
+
(cycle, generation, best_brier, best_roi, best_sharpe, best_composite,
|
| 204 |
+
n_features, model_type, mutation_rate, avg_composite, pop_diversity,
|
| 205 |
+
gen_duration_s, improved)
|
| 206 |
+
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
|
| 207 |
+
(cycle, generation, best["brier"], best["roi"], best["sharpe"],
|
| 208 |
+
best["composite"], best.get("n_features", 0), best.get("model_type", "?"),
|
| 209 |
+
mutation_rate, avg_composite, pop_diversity, duration_s, improved))
|
| 210 |
+
|
| 211 |
+
# Track for auto-cut
|
| 212 |
+
self.brier_history.append(best["brier"])
|
| 213 |
+
if len(self.brier_history) > 50:
|
| 214 |
+
self.brier_history = self.brier_history[-50:]
|
| 215 |
+
|
| 216 |
+
return improved
|
| 217 |
+
|
| 218 |
+
def log_cycle(self, cycle, generation, best, pop_size, mutation_rate, crossover_rate,
|
| 219 |
+
stagnation, games, feature_candidates, cycle_duration_s,
|
| 220 |
+
avg_composite, pop_diversity, top5=None, selected_features=None):
|
| 221 |
+
"""Log one full cycle (multiple generations) result."""
|
| 222 |
+
_exec_sql("""INSERT INTO public.nba_evolution_runs
|
| 223 |
+
(cycle, generation, best_brier, best_roi, best_sharpe, best_calibration,
|
| 224 |
+
best_composite, best_features, best_model_type, pop_size, mutation_rate,
|
| 225 |
+
crossover_rate, stagnation, games, feature_candidates, cycle_duration_s,
|
| 226 |
+
avg_composite, pop_diversity, top5, selected_features)
|
| 227 |
+
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
|
| 228 |
+
(cycle, generation, best["brier"], best["roi"], best["sharpe"],
|
| 229 |
+
best.get("calibration", 0), best["composite"], best.get("n_features", 0),
|
| 230 |
+
best.get("model_type", "?"), pop_size, mutation_rate, crossover_rate,
|
| 231 |
+
stagnation, games, feature_candidates, cycle_duration_s,
|
| 232 |
+
avg_composite, pop_diversity,
|
| 233 |
+
json.dumps(top5 or [], default=str),
|
| 234 |
+
json.dumps(selected_features or [], default=str)))
|
| 235 |
+
|
| 236 |
+
# Local file backup
|
| 237 |
+
entry = {
|
| 238 |
+
"ts": datetime.now(timezone.utc).isoformat(),
|
| 239 |
+
"cycle": cycle, "generation": generation,
|
| 240 |
+
"best": best, "pop_size": pop_size,
|
| 241 |
+
"mutation_rate": mutation_rate, "stagnation": stagnation,
|
| 242 |
+
}
|
| 243 |
+
log_file = self.local_dir / f"cycle-{cycle:04d}.json"
|
| 244 |
+
log_file.write_text(json.dumps(entry, indent=2, default=str))
|
| 245 |
+
|
| 246 |
+
def log_top_evals(self, generation, top_individuals):
|
| 247 |
+
"""Log top 10 individuals for this generation."""
|
| 248 |
+
for rank, ind in enumerate(top_individuals[:10]):
|
| 249 |
+
_exec_sql("""INSERT INTO public.nba_evolution_evals
|
| 250 |
+
(generation, individual_rank, brier, roi, sharpe, composite, n_features, model_type)
|
| 251 |
+
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)""",
|
| 252 |
+
(generation, rank + 1,
|
| 253 |
+
ind.get("brier", ind.get("fitness", {}).get("brier", 0)),
|
| 254 |
+
ind.get("roi", ind.get("fitness", {}).get("roi", 0)),
|
| 255 |
+
ind.get("sharpe", ind.get("fitness", {}).get("sharpe", 0)),
|
| 256 |
+
ind.get("composite", ind.get("fitness", {}).get("composite", 0)),
|
| 257 |
+
ind.get("n_features", 0),
|
| 258 |
+
ind.get("model_type", ind.get("hyperparams", {}).get("model_type", "?"))))
|
| 259 |
+
|
| 260 |
+
def log_cut(self, cut_type, reason, brier_before, brier_after, action, params=None):
|
| 261 |
+
"""Log an auto-cut event."""
|
| 262 |
+
_exec_sql("""INSERT INTO public.nba_evolution_cuts
|
| 263 |
+
(cut_type, reason, brier_before, brier_after, action_taken, params_applied)
|
| 264 |
+
VALUES (%s,%s,%s,%s,%s,%s)""",
|
| 265 |
+
(cut_type, reason, brier_before, brier_after, action,
|
| 266 |
+
json.dumps(params or {}, default=str)))
|
| 267 |
+
self.cuts_applied += 1
|
| 268 |
+
print(f"[AUTO-CUT] {cut_type}: {reason} → {action}")
|
| 269 |
+
|
| 270 |
+
# ══════════════════════════════════════════
|
| 271 |
+
# AUTO-CUT — Automatic regression/stagnation handling
|
| 272 |
+
# ══════════════════════════════════════════
|
| 273 |
+
|
| 274 |
+
def check_auto_cut(self, current_best, engine_state):
|
| 275 |
+
"""
|
| 276 |
+
Check if an auto-cut should be applied.
|
| 277 |
+
Returns: list of actions to take, or empty list.
|
| 278 |
+
|
| 279 |
+
Actions are dicts: {"type": "...", "params": {...}}
|
| 280 |
+
The caller (evolution loop) is responsible for executing them.
|
| 281 |
+
"""
|
| 282 |
+
actions = []
|
| 283 |
+
brier = current_best.get("brier", 1.0)
|
| 284 |
+
composite = current_best.get("composite", 0)
|
| 285 |
+
|
| 286 |
+
# ── RULE 1: REGRESSION CUT ──
|
| 287 |
+
# If Brier is getting worse for 3+ consecutive generations
|
| 288 |
+
if len(self.brier_history) >= 3:
|
| 289 |
+
last3 = self.brier_history[-3:]
|
| 290 |
+
if all(last3[i] > last3[i-1] + 0.0003 for i in range(1, len(last3))):
|
| 291 |
+
self.regression_count += 1
|
| 292 |
+
if self.regression_count >= 2:
|
| 293 |
+
self.log_cut("REGRESSION", f"Brier increasing 3+ gens: {[f'{b:.4f}' for b in last3]}",
|
| 294 |
+
last3[0], last3[-1], "rollback_mutation",
|
| 295 |
+
{"mutation_rate": max(0.03, engine_state.get("mutation_rate", 0.1) * 0.5)})
|
| 296 |
+
actions.append({
|
| 297 |
+
"type": "config",
|
| 298 |
+
"params": {"mutation_rate": max(0.03, engine_state.get("mutation_rate", 0.1) * 0.5)},
|
| 299 |
+
})
|
| 300 |
+
self.regression_count = 0
|
| 301 |
+
else:
|
| 302 |
+
self.regression_count = 0
|
| 303 |
+
|
| 304 |
+
# ── RULE 2: STAGNATION CUT ──
|
| 305 |
+
# At 20+ gens stagnation, diversify moderately (don't destroy good individuals)
|
| 306 |
+
stagnation = engine_state.get("stagnation", 0)
|
| 307 |
+
if stagnation >= 20:
|
| 308 |
+
current_mut = engine_state.get("mutation_rate", 0.04)
|
| 309 |
+
# Boost mutation by 2x (capped at 0.15) — enough to explore without random noise
|
| 310 |
+
new_mut = min(0.15, current_mut * 2)
|
| 311 |
+
self.log_cut("STAGNATION", f"No improvement for {stagnation} gens (mutation {current_mut:.3f} → {new_mut:.3f})",
|
| 312 |
+
brier, brier, "moderate_diversify",
|
| 313 |
+
{"mutation_rate": new_mut})
|
| 314 |
+
actions.append({
|
| 315 |
+
"type": "emergency_diversify",
|
| 316 |
+
"params": {"mutation_rate": new_mut},
|
| 317 |
+
})
|
| 318 |
+
|
| 319 |
+
# ── RULE 3: ROI CUT ──
|
| 320 |
+
roi = current_best.get("roi", 0)
|
| 321 |
+
if roi < -0.15:
|
| 322 |
+
self.log_cut("ROI_THRESHOLD", f"ROI dropped to {roi:.1%} — betting paused",
|
| 323 |
+
brier, brier, "pause_betting")
|
| 324 |
+
# Don't stop evolution, just flag for betting logic
|
| 325 |
+
actions.append({"type": "flag", "params": {"pause_betting": True}})
|
| 326 |
+
|
| 327 |
+
# ── RULE 4: DIVERSITY CUT ──
|
| 328 |
+
diversity = engine_state.get("pop_diversity", 0)
|
| 329 |
+
if diversity < 3.0 and engine_state.get("pop_size", 0) > 20:
|
| 330 |
+
self.log_cut("DIVERSITY", f"Population diversity {diversity:.1f} too low",
|
| 331 |
+
brier, brier, "inject_random",
|
| 332 |
+
{"inject_count": max(10, engine_state.get("pop_size", 50) // 4)})
|
| 333 |
+
actions.append({
|
| 334 |
+
"type": "inject",
|
| 335 |
+
"params": {"count": max(10, engine_state.get("pop_size", 50) // 4)},
|
| 336 |
+
})
|
| 337 |
+
|
| 338 |
+
# ── RULE 5: FEATURE CUT ──
|
| 339 |
+
n_features = current_best.get("n_features", 0)
|
| 340 |
+
if 0 < n_features < 40:
|
| 341 |
+
self.log_cut("LOW_FEATURES", f"Only {n_features} features selected",
|
| 342 |
+
brier, brier, "expand_target",
|
| 343 |
+
{"target_features": 200})
|
| 344 |
+
actions.append({
|
| 345 |
+
"type": "config",
|
| 346 |
+
"params": {"target_features": 200},
|
| 347 |
+
})
|
| 348 |
+
|
| 349 |
+
# ── RULE 6: BRIER FLOOR ──
|
| 350 |
+
# If Brier is stuck above 0.24 for 50+ gens, try moderate exploration (NOT full reset)
|
| 351 |
+
# Old rule was destroying progress: mutation 0.25 = random noise, pop 250 > hard cap 80
|
| 352 |
+
if len(self.brier_history) >= 50:
|
| 353 |
+
if all(b > 0.24 for b in self.brier_history[-50:]):
|
| 354 |
+
self.log_cut("BRIER_FLOOR", f"Brier stuck above 0.24 for 50+ gens (best: {min(self.brier_history[-50:]):.4f})",
|
| 355 |
+
brier, brier, "moderate_diversify",
|
| 356 |
+
{"mutation_rate": 0.12, "target_features": 150})
|
| 357 |
+
actions.append({
|
| 358 |
+
"type": "emergency_diversify",
|
| 359 |
+
"params": {"mutation_rate": 0.12, "target_features": 150},
|
| 360 |
+
})
|
| 361 |
+
# Clear history so rule doesn't fire every single generation
|
| 362 |
+
self.brier_history = self.brier_history[-10:]
|
| 363 |
+
|
| 364 |
+
# ── RULE 7: LIVE REGRESSION CUT ──
|
| 365 |
+
# If live Brier (from daily eval) is worse than best checkpoint + 0.01 → rollback
|
| 366 |
+
live_brier = engine_state.get("live_brier")
|
| 367 |
+
best_cp_brier = engine_state.get("best_checkpoint_brier")
|
| 368 |
+
if live_brier is not None and best_cp_brier is not None:
|
| 369 |
+
if live_brier > best_cp_brier + 0.01:
|
| 370 |
+
self.log_cut("LIVE_REGRESSION",
|
| 371 |
+
f"Live Brier {live_brier:.4f} > checkpoint {best_cp_brier:.4f} + 0.01",
|
| 372 |
+
best_cp_brier, live_brier, "rollback_to_checkpoint",
|
| 373 |
+
{"target_brier": best_cp_brier})
|
| 374 |
+
actions.append({
|
| 375 |
+
"type": "rollback",
|
| 376 |
+
"params": {"reason": "live_regression", "live_brier": live_brier, "checkpoint_brier": best_cp_brier},
|
| 377 |
+
})
|
| 378 |
+
|
| 379 |
+
# Update tracking
|
| 380 |
+
if brier < self.last_best_brier:
|
| 381 |
+
self.last_best_brier = brier
|
| 382 |
+
self.last_best_composite = composite
|
| 383 |
+
|
| 384 |
+
return actions
|
| 385 |
+
|
| 386 |
+
# ══════════════════════════════════════════
|
| 387 |
+
# QUERY — Read logged data
|
| 388 |
+
# ══════════════════════════════════════════
|
| 389 |
+
|
| 390 |
+
def get_recent_runs(self, limit=20):
|
| 391 |
+
"""Get recent cycle logs from Supabase."""
|
| 392 |
+
rows = _exec_sql(
|
| 393 |
+
"SELECT * FROM public.nba_evolution_runs ORDER BY ts DESC LIMIT %s", (limit,))
|
| 394 |
+
return rows or []
|
| 395 |
+
|
| 396 |
+
def get_recent_cuts(self, limit=10):
|
| 397 |
+
rows = _exec_sql(
|
| 398 |
+
"SELECT * FROM public.nba_evolution_cuts ORDER BY ts DESC LIMIT %s", (limit,))
|
| 399 |
+
return rows or []
|
| 400 |
+
|
| 401 |
+
def get_brier_trend(self, last_n=50):
|
| 402 |
+
rows = _exec_sql(
|
| 403 |
+
"SELECT generation, best_brier FROM public.nba_evolution_gens ORDER BY ts DESC LIMIT %s",
|
| 404 |
+
(last_n,))
|
| 405 |
+
if rows:
|
| 406 |
+
return [(r[0], r[1]) for r in reversed(rows)]
|
| 407 |
+
return self.brier_history[-last_n:]
|
| 408 |
+
|
| 409 |
+
def get_stats(self):
|
| 410 |
+
"""Summary stats for dashboard."""
|
| 411 |
+
total_gens = _exec_sql("SELECT COUNT(*) FROM public.nba_evolution_gens")
|
| 412 |
+
total_runs = _exec_sql("SELECT COUNT(*) FROM public.nba_evolution_runs")
|
| 413 |
+
total_cuts = _exec_sql("SELECT COUNT(*) FROM public.nba_evolution_cuts")
|
| 414 |
+
best_ever = _exec_sql(
|
| 415 |
+
"SELECT MIN(best_brier) FROM public.nba_evolution_runs")
|
| 416 |
+
|
| 417 |
+
return {
|
| 418 |
+
"total_generations": total_gens[0][0] if total_gens else 0,
|
| 419 |
+
"total_cycles": total_runs[0][0] if total_runs else 0,
|
| 420 |
+
"total_cuts": total_cuts[0][0] if total_cuts else 0,
|
| 421 |
+
"best_brier_ever": best_ever[0][0] if best_ever and best_ever[0][0] else None,
|
| 422 |
+
"local_cuts_applied": self.cuts_applied,
|
| 423 |
+
"regression_count": self.regression_count,
|
| 424 |
+
"brier_history_len": len(self.brier_history),
|
| 425 |
+
}
|
evolution/sota_s21.py
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""S21 SOTA hooks — Darwinian weights driven by rolling 30-day PnL.
|
| 2 |
+
|
| 3 |
+
Inspiration: atlas-gic (research_cycle7_sota_gap.md).
|
| 4 |
+
|
| 5 |
+
Instead of weighting ensemble members by validation log-loss, we track a
|
| 6 |
+
rolling 30-day PnL per base estimator and recompute weights every 10
|
| 7 |
+
generations with exponential decay alpha=0.9.
|
| 8 |
+
|
| 9 |
+
Usage:
|
| 10 |
+
tracker = DarwinianWeights(n_models=3, window_days=30, alpha=0.9,
|
| 11 |
+
reweight_every=10)
|
| 12 |
+
tracker.record_pnl(model_idx=0, date="2026-04-15", pnl=12.3)
|
| 13 |
+
...
|
| 14 |
+
if tracker.should_reweight(current_gen):
|
| 15 |
+
weights = tracker.compute_weights() # -> np.ndarray shape (n_models,)
|
| 16 |
+
final_prob = np.dot(weights, [p0, p1, p2])
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from __future__ import annotations
|
| 20 |
+
|
| 21 |
+
from collections import deque
|
| 22 |
+
from dataclasses import dataclass, field
|
| 23 |
+
from datetime import datetime
|
| 24 |
+
from typing import Deque, Dict, List
|
| 25 |
+
|
| 26 |
+
import numpy as np
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
@dataclass
|
| 30 |
+
class DarwinianWeights:
|
| 31 |
+
n_models: int
|
| 32 |
+
window_days: int = 30
|
| 33 |
+
alpha: float = 0.9
|
| 34 |
+
reweight_every: int = 10
|
| 35 |
+
_history: Dict[int, Deque] = field(default_factory=dict)
|
| 36 |
+
_cached_weights: np.ndarray = field(default=None)
|
| 37 |
+
_last_reweight_gen: int = -1
|
| 38 |
+
|
| 39 |
+
def __post_init__(self):
|
| 40 |
+
self._history = {i: deque() for i in range(self.n_models)}
|
| 41 |
+
self._cached_weights = np.ones(self.n_models) / self.n_models
|
| 42 |
+
|
| 43 |
+
def record_pnl(self, model_idx: int, date: str, pnl: float) -> None:
|
| 44 |
+
"""Record a single-day PnL contribution for a base model."""
|
| 45 |
+
if model_idx < 0 or model_idx >= self.n_models:
|
| 46 |
+
return
|
| 47 |
+
try:
|
| 48 |
+
d = datetime.fromisoformat(date[:10])
|
| 49 |
+
except Exception:
|
| 50 |
+
d = datetime.utcnow()
|
| 51 |
+
hist = self._history[model_idx]
|
| 52 |
+
hist.append((d, float(pnl)))
|
| 53 |
+
# trim to window
|
| 54 |
+
cutoff = datetime.utcnow().toordinal() - self.window_days
|
| 55 |
+
while hist and hist[0][0].toordinal() < cutoff:
|
| 56 |
+
hist.popleft()
|
| 57 |
+
|
| 58 |
+
def should_reweight(self, current_gen: int) -> bool:
|
| 59 |
+
return (current_gen - self._last_reweight_gen) >= self.reweight_every
|
| 60 |
+
|
| 61 |
+
def compute_weights(self) -> np.ndarray:
|
| 62 |
+
"""Compute softmax weights across models based on exponentially
|
| 63 |
+
decayed rolling PnL. Returns a normalized weight vector."""
|
| 64 |
+
scores = np.zeros(self.n_models)
|
| 65 |
+
for i in range(self.n_models):
|
| 66 |
+
hist = list(self._history[i])
|
| 67 |
+
if not hist:
|
| 68 |
+
scores[i] = 0.0
|
| 69 |
+
continue
|
| 70 |
+
# Sort ascending by date, apply exponential decay from most recent.
|
| 71 |
+
hist.sort(key=lambda t: t[0])
|
| 72 |
+
# Walk newest -> oldest with decay
|
| 73 |
+
weighted = 0.0
|
| 74 |
+
w = 1.0
|
| 75 |
+
for d, pnl in reversed(hist):
|
| 76 |
+
weighted += w * pnl
|
| 77 |
+
w *= self.alpha
|
| 78 |
+
scores[i] = weighted
|
| 79 |
+
|
| 80 |
+
# Softmax with temperature to keep weights bounded
|
| 81 |
+
if np.all(scores == 0):
|
| 82 |
+
w = np.ones(self.n_models) / self.n_models
|
| 83 |
+
else:
|
| 84 |
+
temp = max(1e-3, float(np.std(scores)))
|
| 85 |
+
z = scores / temp
|
| 86 |
+
z -= z.max()
|
| 87 |
+
exp_z = np.exp(z)
|
| 88 |
+
w = exp_z / exp_z.sum()
|
| 89 |
+
self._cached_weights = w
|
| 90 |
+
return w
|
| 91 |
+
|
| 92 |
+
def get_weights(self, current_gen: int = 0) -> np.ndarray:
|
| 93 |
+
"""Return cached weights; recompute if reweight interval hit."""
|
| 94 |
+
if self.should_reweight(current_gen):
|
| 95 |
+
self._last_reweight_gen = current_gen
|
| 96 |
+
return self.compute_weights()
|
| 97 |
+
return self._cached_weights
|
| 98 |
+
|
| 99 |
+
def snapshot(self) -> dict:
|
| 100 |
+
return {
|
| 101 |
+
"n_models": self.n_models,
|
| 102 |
+
"window_days": self.window_days,
|
| 103 |
+
"alpha": self.alpha,
|
| 104 |
+
"reweight_every": self.reweight_every,
|
| 105 |
+
"last_reweight_gen": self._last_reweight_gen,
|
| 106 |
+
"weights": self._cached_weights.tolist(),
|
| 107 |
+
"history_counts": {i: len(self._history[i]) for i in range(self.n_models)},
|
| 108 |
+
}
|
experiment_runner.py
ADDED
|
@@ -0,0 +1,1073 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
S11 Experiment Runner — Isolated Evaluation Server
|
| 4 |
+
=====================================================
|
| 5 |
+
Turns S11 from a clone of S10 into a dedicated experiment server.
|
| 6 |
+
Agents (Eve, CrewAI, etc.) submit experiments to Supabase queue.
|
| 7 |
+
S11 polls the queue, evaluates in isolation (walk-forward backtest),
|
| 8 |
+
and stores results back. S10's population is NEVER touched.
|
| 9 |
+
|
| 10 |
+
Experiment Types:
|
| 11 |
+
- feature_test: Test specific feature mask → evaluate with walk-forward
|
| 12 |
+
- model_test: Test specific model_type + hyperparams → evaluate
|
| 13 |
+
- calibration_test: Test calibration method on current best features
|
| 14 |
+
- config_change: Test GA config by running mini-evolution (5 gens)
|
| 15 |
+
|
| 16 |
+
Queue: Supabase table `nba_experiments`
|
| 17 |
+
status: pending → running → completed | failed
|
| 18 |
+
"""
|
| 19 |
+
|
| 20 |
+
import os
|
| 21 |
+
import sys
|
| 22 |
+
import json
|
| 23 |
+
import time
|
| 24 |
+
import random
|
| 25 |
+
import traceback
|
| 26 |
+
import threading
|
| 27 |
+
import numpy as np
|
| 28 |
+
from datetime import datetime, timezone
|
| 29 |
+
from pathlib import Path
|
| 30 |
+
from typing import Optional, Dict, Any
|
| 31 |
+
|
| 32 |
+
# ── Import shared functions from app.py (lazy — app.py must be loaded first) ──
|
| 33 |
+
# When this module is imported from app.py's bottom section, app is already
|
| 34 |
+
# fully loaded in sys.modules. Direct `from app import` would re-execute
|
| 35 |
+
# app.py's top-level code (Gradio, data loading) causing a hang.
|
| 36 |
+
# Instead, we grab references AFTER import, in init_from_app().
|
| 37 |
+
|
| 38 |
+
def _default_log(msg, level="INFO"):
|
| 39 |
+
print(f"[{level}] {msg}")
|
| 40 |
+
|
| 41 |
+
Individual = None
|
| 42 |
+
evaluate = None
|
| 43 |
+
_build = None
|
| 44 |
+
_prune_correlated_features = None
|
| 45 |
+
_log_loss_score = None
|
| 46 |
+
_ece = None
|
| 47 |
+
_evaluate_stacking = None
|
| 48 |
+
load_all_games = None
|
| 49 |
+
build_features = None
|
| 50 |
+
pull_seasons = None
|
| 51 |
+
log = _default_log
|
| 52 |
+
live = {}
|
| 53 |
+
FAST_EVAL_GAMES = 7000
|
| 54 |
+
DATA_DIR = Path("/data") if Path("/data").exists() else Path("data")
|
| 55 |
+
HIST_DIR = DATA_DIR / "historical"
|
| 56 |
+
STATE_DIR = DATA_DIR / "evolution-state"
|
| 57 |
+
RESULTS_DIR = DATA_DIR / "results"
|
| 58 |
+
|
| 59 |
+
def init_from_app():
|
| 60 |
+
"""Grab references from the already-loaded app module. Call once at startup."""
|
| 61 |
+
import sys as _sys
|
| 62 |
+
app_mod = _sys.modules.get('app') or _sys.modules.get('__main__')
|
| 63 |
+
if app_mod is None:
|
| 64 |
+
raise RuntimeError("app module not loaded")
|
| 65 |
+
|
| 66 |
+
global Individual, evaluate, _build, _prune_correlated_features
|
| 67 |
+
global _log_loss_score, _ece, _evaluate_stacking
|
| 68 |
+
global load_all_games, build_features, pull_seasons, log, live
|
| 69 |
+
global FAST_EVAL_GAMES, DATA_DIR, HIST_DIR, STATE_DIR, RESULTS_DIR
|
| 70 |
+
|
| 71 |
+
Individual = getattr(app_mod, 'Individual')
|
| 72 |
+
evaluate = getattr(app_mod, 'evaluate')
|
| 73 |
+
_build = getattr(app_mod, '_build')
|
| 74 |
+
_prune_correlated_features = getattr(app_mod, '_prune_correlated_features')
|
| 75 |
+
_log_loss_score = getattr(app_mod, '_log_loss_score')
|
| 76 |
+
_ece = getattr(app_mod, '_ece')
|
| 77 |
+
_evaluate_stacking = getattr(app_mod, '_evaluate_stacking')
|
| 78 |
+
load_all_games = getattr(app_mod, 'load_all_games')
|
| 79 |
+
build_features = getattr(app_mod, 'build_features')
|
| 80 |
+
pull_seasons = getattr(app_mod, 'pull_seasons')
|
| 81 |
+
log = getattr(app_mod, 'log')
|
| 82 |
+
live = getattr(app_mod, 'live')
|
| 83 |
+
FAST_EVAL_GAMES = getattr(app_mod, 'FAST_EVAL_GAMES', 7000)
|
| 84 |
+
DATA_DIR = getattr(app_mod, 'DATA_DIR', DATA_DIR)
|
| 85 |
+
HIST_DIR = getattr(app_mod, 'HIST_DIR', HIST_DIR)
|
| 86 |
+
STATE_DIR = getattr(app_mod, 'STATE_DIR', STATE_DIR)
|
| 87 |
+
RESULTS_DIR = getattr(app_mod, 'RESULTS_DIR', RESULTS_DIR)
|
| 88 |
+
|
| 89 |
+
# ── Constants ──
|
| 90 |
+
POLL_INTERVAL = 60 # Poll Supabase every 60 seconds
|
| 91 |
+
MAX_EVAL_GAMES = 7000 # Cap evaluation to prevent OOM (16GB Space)
|
| 92 |
+
MINI_EVO_GENS = 5 # Generations for config_change experiments
|
| 93 |
+
MINI_EVO_POP = 20 # Small population for config_change experiments
|
| 94 |
+
EXPERIMENT_TIMEOUT = 1800 # 30 min max per experiment
|
| 95 |
+
|
| 96 |
+
# ── Supabase connection (same pattern as run_logger.py) ──
|
| 97 |
+
_pg_pool = None
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
def _get_pg():
|
| 101 |
+
"""Lazy PostgreSQL connection pool for Supabase."""
|
| 102 |
+
global _pg_pool
|
| 103 |
+
if _pg_pool is not None:
|
| 104 |
+
return _pg_pool
|
| 105 |
+
db_url = os.environ.get("DATABASE_URL", "")
|
| 106 |
+
if not db_url:
|
| 107 |
+
log("[EXPERIMENT] DATABASE_URL not set — cannot connect to Supabase", "ERROR")
|
| 108 |
+
return None
|
| 109 |
+
try:
|
| 110 |
+
import psycopg2
|
| 111 |
+
from psycopg2 import pool as pg_pool
|
| 112 |
+
_pg_pool = pg_pool.SimpleConnectionPool(1, 3, db_url, options="-c search_path=public")
|
| 113 |
+
conn = _pg_pool.getconn()
|
| 114 |
+
with conn.cursor() as cur:
|
| 115 |
+
cur.execute("SELECT 1")
|
| 116 |
+
_pg_pool.putconn(conn)
|
| 117 |
+
log("[EXPERIMENT] PostgreSQL connected to Supabase OK")
|
| 118 |
+
return _pg_pool
|
| 119 |
+
except Exception as e:
|
| 120 |
+
log(f"[EXPERIMENT] PostgreSQL connection failed: {e}", "ERROR")
|
| 121 |
+
_pg_pool = None
|
| 122 |
+
return None
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
def _exec_sql(sql, params=None, fetch=True):
|
| 126 |
+
"""Execute SQL on Supabase. Returns rows on SELECT, True on INSERT/UPDATE, None on failure."""
|
| 127 |
+
pool = _get_pg()
|
| 128 |
+
if not pool:
|
| 129 |
+
return None
|
| 130 |
+
conn = None
|
| 131 |
+
try:
|
| 132 |
+
conn = pool.getconn()
|
| 133 |
+
with conn.cursor() as cur:
|
| 134 |
+
cur.execute(sql, params)
|
| 135 |
+
conn.commit()
|
| 136 |
+
if fetch:
|
| 137 |
+
try:
|
| 138 |
+
return cur.fetchall()
|
| 139 |
+
except Exception:
|
| 140 |
+
return True
|
| 141 |
+
return True
|
| 142 |
+
except Exception as e:
|
| 143 |
+
log(f"[EXPERIMENT] SQL error: {e}", "ERROR")
|
| 144 |
+
if conn:
|
| 145 |
+
try:
|
| 146 |
+
conn.rollback()
|
| 147 |
+
except Exception:
|
| 148 |
+
pass
|
| 149 |
+
return None
|
| 150 |
+
finally:
|
| 151 |
+
if conn and pool:
|
| 152 |
+
try:
|
| 153 |
+
pool.putconn(conn)
|
| 154 |
+
except Exception:
|
| 155 |
+
pass
|
| 156 |
+
|
| 157 |
+
|
| 158 |
+
def _reconnect_pg():
|
| 159 |
+
"""Force reconnection on next call (e.g., after connection timeout)."""
|
| 160 |
+
global _pg_pool
|
| 161 |
+
if _pg_pool:
|
| 162 |
+
try:
|
| 163 |
+
_pg_pool.closeall()
|
| 164 |
+
except Exception:
|
| 165 |
+
pass
|
| 166 |
+
_pg_pool = None
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
# ═══════════════════════════════════════════════════════
|
| 170 |
+
# EXPERIMENT FETCHER
|
| 171 |
+
# ═══════════════════════════════════════════════════════
|
| 172 |
+
|
| 173 |
+
def fetch_next_experiment() -> Optional[Dict[str, Any]]:
|
| 174 |
+
"""Fetch AND atomically claim the next pending experiment from Supabase.
|
| 175 |
+
|
| 176 |
+
Returns dict with all columns, or None if queue empty.
|
| 177 |
+
Uses CTE with FOR UPDATE SKIP LOCKED to prevent double-pickup.
|
| 178 |
+
"""
|
| 179 |
+
rows = _exec_sql("""
|
| 180 |
+
WITH next_exp AS (
|
| 181 |
+
SELECT id FROM public.nba_experiments
|
| 182 |
+
WHERE status = 'pending'
|
| 183 |
+
AND (target_space IS NULL OR target_space = 'S11' OR target_space = 'any')
|
| 184 |
+
ORDER BY priority DESC, created_at ASC
|
| 185 |
+
LIMIT 1
|
| 186 |
+
FOR UPDATE SKIP LOCKED
|
| 187 |
+
)
|
| 188 |
+
UPDATE public.nba_experiments e
|
| 189 |
+
SET status = 'running', started_at = NOW()
|
| 190 |
+
FROM next_exp
|
| 191 |
+
WHERE e.id = next_exp.id
|
| 192 |
+
RETURNING e.id, e.experiment_id, e.agent_name, e.experiment_type,
|
| 193 |
+
e.description, e.hypothesis, e.params, e.priority,
|
| 194 |
+
e.status, e.target_space, e.baseline_brier, e.created_at
|
| 195 |
+
""")
|
| 196 |
+
if not rows or rows is True or len(rows) == 0:
|
| 197 |
+
return None
|
| 198 |
+
row = rows[0]
|
| 199 |
+
return {
|
| 200 |
+
"id": row[0],
|
| 201 |
+
"experiment_id": row[1],
|
| 202 |
+
"agent_name": row[2],
|
| 203 |
+
"experiment_type": row[3],
|
| 204 |
+
"description": row[4],
|
| 205 |
+
"hypothesis": row[5],
|
| 206 |
+
"params": row[6] if isinstance(row[6], dict) else json.loads(row[6]) if row[6] else {},
|
| 207 |
+
"priority": row[7],
|
| 208 |
+
"status": row[8],
|
| 209 |
+
"target_space": row[9],
|
| 210 |
+
"baseline_brier": row[10],
|
| 211 |
+
"created_at": str(row[11]) if row[11] else None,
|
| 212 |
+
}
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
def claim_experiment(exp_id: int) -> bool:
|
| 216 |
+
"""Legacy claim — now handled atomically in fetch_next_experiment(), kept for compatibility."""
|
| 217 |
+
return True
|
| 218 |
+
|
| 219 |
+
|
| 220 |
+
def _sanitize_for_json(obj):
|
| 221 |
+
"""Convert numpy types to native Python for JSON serialization."""
|
| 222 |
+
if isinstance(obj, dict):
|
| 223 |
+
return {k: _sanitize_for_json(v) for k, v in obj.items()}
|
| 224 |
+
if isinstance(obj, (list, tuple)):
|
| 225 |
+
return [_sanitize_for_json(v) for v in obj]
|
| 226 |
+
if hasattr(obj, 'item'): # numpy scalar
|
| 227 |
+
return obj.item()
|
| 228 |
+
return obj
|
| 229 |
+
|
| 230 |
+
|
| 231 |
+
def complete_experiment(exp_id: int, brier: float, accuracy: float,
|
| 232 |
+
log_loss_val: float, details: dict, status: str = "completed"):
|
| 233 |
+
"""Write results back to Supabase."""
|
| 234 |
+
clean_details = _sanitize_for_json(details)
|
| 235 |
+
_exec_sql("""
|
| 236 |
+
UPDATE public.nba_experiments
|
| 237 |
+
SET status = %s,
|
| 238 |
+
result_brier = %s,
|
| 239 |
+
result_accuracy = %s,
|
| 240 |
+
result_log_loss = %s,
|
| 241 |
+
result_details = %s,
|
| 242 |
+
feature_engine_version = %s,
|
| 243 |
+
completed_at = NOW()
|
| 244 |
+
WHERE id = %s
|
| 245 |
+
""", (status, float(brier), float(accuracy), float(log_loss_val),
|
| 246 |
+
json.dumps(clean_details), "v3.0-35cat-6000feat", int(exp_id)), fetch=False)
|
| 247 |
+
|
| 248 |
+
|
| 249 |
+
def fail_experiment(exp_id: int, error_msg: str):
|
| 250 |
+
"""Mark experiment as failed with error details."""
|
| 251 |
+
details = {"error": error_msg[:2000], "failed_at": datetime.now(timezone.utc).isoformat()}
|
| 252 |
+
_exec_sql("""
|
| 253 |
+
UPDATE public.nba_experiments
|
| 254 |
+
SET status = 'failed',
|
| 255 |
+
result_details = %s,
|
| 256 |
+
completed_at = NOW()
|
| 257 |
+
WHERE id = %s
|
| 258 |
+
""", (json.dumps(details), exp_id), fetch=False)
|
| 259 |
+
|
| 260 |
+
|
| 261 |
+
# ═══════════════════════════════════════════════════════
|
| 262 |
+
# EXPERIMENT EXECUTORS
|
| 263 |
+
# ═══════════════════════════════════════════════════════
|
| 264 |
+
|
| 265 |
+
def _make_individual(n_features: int, params: dict) -> Individual:
|
| 266 |
+
"""Create an Individual from experiment params.
|
| 267 |
+
|
| 268 |
+
params can contain:
|
| 269 |
+
- features: list of ints (feature mask) or list of feature indices
|
| 270 |
+
- feature_indices: list of ints (indices to enable)
|
| 271 |
+
- hyperparams: dict of hyperparams (merged with defaults)
|
| 272 |
+
- model_type: str (shortcut for hyperparams.model_type)
|
| 273 |
+
- calibration: str (shortcut for hyperparams.calibration)
|
| 274 |
+
"""
|
| 275 |
+
ind = Individual(n_features, target=params.get("target_features", 80))
|
| 276 |
+
|
| 277 |
+
# Feature mask: explicit list
|
| 278 |
+
if "features" in params:
|
| 279 |
+
feat = params["features"]
|
| 280 |
+
if len(feat) == n_features:
|
| 281 |
+
ind.features = [int(f) for f in feat]
|
| 282 |
+
else:
|
| 283 |
+
# Treat as indices
|
| 284 |
+
ind.features = [0] * n_features
|
| 285 |
+
for idx in feat:
|
| 286 |
+
if 0 <= idx < n_features:
|
| 287 |
+
ind.features[idx] = 1
|
| 288 |
+
|
| 289 |
+
# Feature indices: explicit list of which features to enable
|
| 290 |
+
if "feature_indices" in params:
|
| 291 |
+
ind.features = [0] * n_features
|
| 292 |
+
for idx in params["feature_indices"]:
|
| 293 |
+
if 0 <= idx < n_features:
|
| 294 |
+
ind.features[idx] = 1
|
| 295 |
+
|
| 296 |
+
# Hyperparams: merge with defaults
|
| 297 |
+
if "hyperparams" in params:
|
| 298 |
+
for k, v in params["hyperparams"].items():
|
| 299 |
+
if k in ind.hyperparams:
|
| 300 |
+
ind.hyperparams[k] = v
|
| 301 |
+
|
| 302 |
+
# Shortcuts
|
| 303 |
+
if "model_type" in params:
|
| 304 |
+
ind.hyperparams["model_type"] = params["model_type"]
|
| 305 |
+
if "calibration" in params:
|
| 306 |
+
ind.hyperparams["calibration"] = params["calibration"]
|
| 307 |
+
|
| 308 |
+
ind.n_features = sum(ind.features)
|
| 309 |
+
return ind
|
| 310 |
+
|
| 311 |
+
|
| 312 |
+
def run_feature_test(experiment: dict, X: np.ndarray, y: np.ndarray,
|
| 313 |
+
feature_names: list) -> dict:
|
| 314 |
+
"""Test specific feature configuration with walk-forward evaluation.
|
| 315 |
+
|
| 316 |
+
params should contain:
|
| 317 |
+
- features or feature_indices: which features to enable
|
| 318 |
+
- hyperparams (optional): model hyperparams
|
| 319 |
+
- model_type (optional): which model to use (default: xgboost)
|
| 320 |
+
- n_splits (optional): number of walk-forward splits (default: 3)
|
| 321 |
+
"""
|
| 322 |
+
params = experiment["params"]
|
| 323 |
+
n_features = X.shape[1]
|
| 324 |
+
|
| 325 |
+
ind = _make_individual(n_features, params)
|
| 326 |
+
|
| 327 |
+
# If no explicit features specified, use current best's feature set if available
|
| 328 |
+
if "features" not in params and "feature_indices" not in params:
|
| 329 |
+
# Load best individual from state
|
| 330 |
+
best = _load_best_individual(n_features)
|
| 331 |
+
if best:
|
| 332 |
+
ind.features = list(best.features)
|
| 333 |
+
ind.n_features = sum(ind.features)
|
| 334 |
+
|
| 335 |
+
n_splits = params.get("n_splits", 3)
|
| 336 |
+
fast = params.get("fast", False)
|
| 337 |
+
|
| 338 |
+
log(f"[EXPERIMENT] feature_test: {ind.n_features} features, model={ind.hyperparams['model_type']}, splits={n_splits}")
|
| 339 |
+
|
| 340 |
+
# Evaluate with full data (not fast mode by default for experiments)
|
| 341 |
+
evaluate(ind, X, y, n_splits=n_splits, fast=fast)
|
| 342 |
+
|
| 343 |
+
return {
|
| 344 |
+
"brier": ind.fitness.get("brier", 1.0),
|
| 345 |
+
"roi": ind.fitness.get("roi", 0.0),
|
| 346 |
+
"sharpe": ind.fitness.get("sharpe", 0.0),
|
| 347 |
+
"calibration": ind.fitness.get("calibration", 1.0),
|
| 348 |
+
"composite": ind.fitness.get("composite", 0.0),
|
| 349 |
+
"features_pruned": ind.fitness.get("features_pruned", 0),
|
| 350 |
+
"n_features_selected": ind.n_features,
|
| 351 |
+
"model_type": ind.hyperparams["model_type"],
|
| 352 |
+
"hyperparams": ind.hyperparams,
|
| 353 |
+
}
|
| 354 |
+
|
| 355 |
+
|
| 356 |
+
def run_model_test(experiment: dict, X: np.ndarray, y: np.ndarray,
|
| 357 |
+
feature_names: list) -> dict:
|
| 358 |
+
"""Test specific model type and hyperparams.
|
| 359 |
+
|
| 360 |
+
params should contain:
|
| 361 |
+
- model_type: str (xgboost, lightgbm, catboost, random_forest, extra_trees, stacking, mlp)
|
| 362 |
+
- hyperparams (optional): full or partial hyperparams dict
|
| 363 |
+
- features or feature_indices (optional): use specific features
|
| 364 |
+
"""
|
| 365 |
+
params = experiment["params"]
|
| 366 |
+
n_features = X.shape[1]
|
| 367 |
+
|
| 368 |
+
# Start from best individual's features (most fair comparison)
|
| 369 |
+
best = _load_best_individual(n_features)
|
| 370 |
+
ind = _make_individual(n_features, params)
|
| 371 |
+
|
| 372 |
+
if best and "features" not in params and "feature_indices" not in params:
|
| 373 |
+
ind.features = list(best.features)
|
| 374 |
+
ind.n_features = sum(ind.features)
|
| 375 |
+
|
| 376 |
+
# Must have model_type
|
| 377 |
+
if "model_type" not in params:
|
| 378 |
+
raise ValueError("model_test requires 'model_type' in params")
|
| 379 |
+
|
| 380 |
+
ind.hyperparams["model_type"] = params["model_type"]
|
| 381 |
+
n_splits = params.get("n_splits", 3)
|
| 382 |
+
fast = params.get("fast", False)
|
| 383 |
+
|
| 384 |
+
log(f"[EXPERIMENT] model_test: model={ind.hyperparams['model_type']}, "
|
| 385 |
+
f"{ind.n_features} features, splits={n_splits}")
|
| 386 |
+
|
| 387 |
+
evaluate(ind, X, y, n_splits=n_splits, fast=fast)
|
| 388 |
+
|
| 389 |
+
return {
|
| 390 |
+
"brier": ind.fitness.get("brier", 1.0),
|
| 391 |
+
"roi": ind.fitness.get("roi", 0.0),
|
| 392 |
+
"sharpe": ind.fitness.get("sharpe", 0.0),
|
| 393 |
+
"calibration": ind.fitness.get("calibration", 1.0),
|
| 394 |
+
"composite": ind.fitness.get("composite", 0.0),
|
| 395 |
+
"features_pruned": ind.fitness.get("features_pruned", 0),
|
| 396 |
+
"n_features_selected": ind.n_features,
|
| 397 |
+
"model_type": ind.hyperparams["model_type"],
|
| 398 |
+
"hyperparams": ind.hyperparams,
|
| 399 |
+
}
|
| 400 |
+
|
| 401 |
+
|
| 402 |
+
def run_calibration_test(experiment: dict, X: np.ndarray, y: np.ndarray,
|
| 403 |
+
feature_names: list) -> dict:
|
| 404 |
+
"""Test calibration method on current best features.
|
| 405 |
+
|
| 406 |
+
params should contain:
|
| 407 |
+
- calibration: str (isotonic, sigmoid, none)
|
| 408 |
+
- model_type (optional): override model type
|
| 409 |
+
"""
|
| 410 |
+
params = experiment["params"]
|
| 411 |
+
n_features = X.shape[1]
|
| 412 |
+
|
| 413 |
+
best = _load_best_individual(n_features)
|
| 414 |
+
if not best:
|
| 415 |
+
raise ValueError("No best individual found — cannot run calibration_test without baseline features")
|
| 416 |
+
|
| 417 |
+
# Test each calibration method if "all" requested, otherwise just the one specified
|
| 418 |
+
calibration = params.get("calibration", "isotonic")
|
| 419 |
+
methods_to_test = ["isotonic", "sigmoid", "none"] if calibration == "all" else [calibration]
|
| 420 |
+
|
| 421 |
+
results = {}
|
| 422 |
+
best_brier = 1.0
|
| 423 |
+
best_method = None
|
| 424 |
+
|
| 425 |
+
for method in methods_to_test:
|
| 426 |
+
ind = Individual.__new__(Individual)
|
| 427 |
+
ind.features = list(best.features)
|
| 428 |
+
ind.hyperparams = dict(best.hyperparams)
|
| 429 |
+
ind.fitness = {"brier": 1.0, "roi": 0.0, "sharpe": 0.0, "calibration": 1.0, "composite": 0.0}
|
| 430 |
+
ind.generation = 0
|
| 431 |
+
ind.n_features = sum(ind.features)
|
| 432 |
+
ind.hyperparams["calibration"] = method
|
| 433 |
+
|
| 434 |
+
if "model_type" in params:
|
| 435 |
+
ind.hyperparams["model_type"] = params["model_type"]
|
| 436 |
+
|
| 437 |
+
n_splits = params.get("n_splits", 3)
|
| 438 |
+
log(f"[EXPERIMENT] calibration_test: method={method}, model={ind.hyperparams['model_type']}")
|
| 439 |
+
|
| 440 |
+
evaluate(ind, X, y, n_splits=n_splits, fast=False)
|
| 441 |
+
|
| 442 |
+
results[method] = {
|
| 443 |
+
"brier": ind.fitness.get("brier", 1.0),
|
| 444 |
+
"roi": ind.fitness.get("roi", 0.0),
|
| 445 |
+
"sharpe": ind.fitness.get("sharpe", 0.0),
|
| 446 |
+
"calibration": ind.fitness.get("calibration", 1.0),
|
| 447 |
+
"composite": ind.fitness.get("composite", 0.0),
|
| 448 |
+
}
|
| 449 |
+
|
| 450 |
+
if ind.fitness.get("brier", 1.0) < best_brier:
|
| 451 |
+
best_brier = ind.fitness["brier"]
|
| 452 |
+
best_method = method
|
| 453 |
+
|
| 454 |
+
return {
|
| 455 |
+
"brier": best_brier,
|
| 456 |
+
"best_method": best_method,
|
| 457 |
+
"all_results": results,
|
| 458 |
+
"n_features_selected": best.n_features,
|
| 459 |
+
"model_type": best.hyperparams.get("model_type", "unknown"),
|
| 460 |
+
}
|
| 461 |
+
|
| 462 |
+
|
| 463 |
+
def run_config_change(experiment: dict, X: np.ndarray, y: np.ndarray,
|
| 464 |
+
feature_names: list) -> dict:
|
| 465 |
+
"""Test GA config change by running a mini-evolution.
|
| 466 |
+
|
| 467 |
+
params should contain any of:
|
| 468 |
+
- pop_size: int (default 20)
|
| 469 |
+
- mutation_rate: float
|
| 470 |
+
- crossover_rate: float
|
| 471 |
+
- tournament_size: int
|
| 472 |
+
- target_features: int
|
| 473 |
+
- n_generations: int (default 5)
|
| 474 |
+
- elite_size: int
|
| 475 |
+
"""
|
| 476 |
+
params = experiment["params"]
|
| 477 |
+
n_features = X.shape[1]
|
| 478 |
+
|
| 479 |
+
pop_size = min(params.get("pop_size", MINI_EVO_POP), 30) # Cap at 30
|
| 480 |
+
n_gens = min(params.get("n_generations", MINI_EVO_GENS), 10) # Cap at 10
|
| 481 |
+
mutation_rate = params.get("mutation_rate", 0.04)
|
| 482 |
+
crossover_rate = params.get("crossover_rate", 0.80)
|
| 483 |
+
tournament_size = min(params.get("tournament_size", 4), pop_size // 2)
|
| 484 |
+
target_features = params.get("target_features", 80)
|
| 485 |
+
elite_size = min(params.get("elite_size", 3), pop_size // 3)
|
| 486 |
+
|
| 487 |
+
log(f"[EXPERIMENT] config_change: pop={pop_size}, gens={n_gens}, "
|
| 488 |
+
f"mut={mutation_rate}, cx={crossover_rate}, target_feat={target_features}")
|
| 489 |
+
|
| 490 |
+
# Initialize population
|
| 491 |
+
population = []
|
| 492 |
+
model_types = ["xgboost", "lightgbm", "catboost", "random_forest", "extra_trees", "stacking", "mlp"]
|
| 493 |
+
for i in range(pop_size):
|
| 494 |
+
ind = Individual(n_features, target=target_features)
|
| 495 |
+
ind.hyperparams["model_type"] = model_types[i % len(model_types)]
|
| 496 |
+
population.append(ind)
|
| 497 |
+
|
| 498 |
+
# Seed with best individual if available
|
| 499 |
+
best = _load_best_individual(n_features)
|
| 500 |
+
if best and len(population) > 0:
|
| 501 |
+
population[0] = best
|
| 502 |
+
|
| 503 |
+
history = []
|
| 504 |
+
best_ever_brier = 1.0
|
| 505 |
+
best_ever_composite = 0.0
|
| 506 |
+
|
| 507 |
+
for gen in range(n_gens):
|
| 508 |
+
# Evaluate
|
| 509 |
+
for ind in population:
|
| 510 |
+
evaluate(ind, X, y, n_splits=2, fast=True)
|
| 511 |
+
|
| 512 |
+
# Sort by composite
|
| 513 |
+
population.sort(key=lambda x: x.fitness.get("composite", 0), reverse=True)
|
| 514 |
+
gen_best = population[0]
|
| 515 |
+
|
| 516 |
+
gen_brier = gen_best.fitness.get("brier", 1.0)
|
| 517 |
+
gen_composite = gen_best.fitness.get("composite", 0.0)
|
| 518 |
+
if gen_brier < best_ever_brier:
|
| 519 |
+
best_ever_brier = gen_brier
|
| 520 |
+
if gen_composite > best_ever_composite:
|
| 521 |
+
best_ever_composite = gen_composite
|
| 522 |
+
|
| 523 |
+
history.append({
|
| 524 |
+
"generation": gen,
|
| 525 |
+
"best_brier": gen_brier,
|
| 526 |
+
"best_composite": gen_composite,
|
| 527 |
+
"avg_brier": round(np.mean([p.fitness.get("brier", 1.0) for p in population]), 5),
|
| 528 |
+
"avg_composite": round(np.mean([p.fitness.get("composite", 0) for p in population]), 5),
|
| 529 |
+
})
|
| 530 |
+
|
| 531 |
+
log(f"[EXPERIMENT] config_change gen {gen}: best_brier={gen_brier:.4f}, "
|
| 532 |
+
f"composite={gen_composite:.4f}")
|
| 533 |
+
|
| 534 |
+
# Selection + Crossover + Mutation (mini-GA)
|
| 535 |
+
elites = population[:elite_size]
|
| 536 |
+
new_pop = list(elites)
|
| 537 |
+
|
| 538 |
+
while len(new_pop) < pop_size:
|
| 539 |
+
# Tournament selection
|
| 540 |
+
candidates = random.sample(population, min(tournament_size, len(population)))
|
| 541 |
+
p1 = max(candidates, key=lambda x: x.fitness.get("composite", 0))
|
| 542 |
+
candidates = random.sample(population, min(tournament_size, len(population)))
|
| 543 |
+
p2 = max(candidates, key=lambda x: x.fitness.get("composite", 0))
|
| 544 |
+
|
| 545 |
+
if random.random() < crossover_rate:
|
| 546 |
+
child = Individual.crossover(p1, p2)
|
| 547 |
+
else:
|
| 548 |
+
child = Individual(n_features, target=target_features)
|
| 549 |
+
|
| 550 |
+
child.mutate(rate=mutation_rate)
|
| 551 |
+
new_pop.append(child)
|
| 552 |
+
|
| 553 |
+
population = new_pop[:pop_size]
|
| 554 |
+
|
| 555 |
+
# Final evaluation (full, not fast)
|
| 556 |
+
final_best = max(population, key=lambda x: x.fitness.get("composite", 0))
|
| 557 |
+
evaluate(final_best, X, y, n_splits=3, fast=False)
|
| 558 |
+
|
| 559 |
+
return {
|
| 560 |
+
"brier": final_best.fitness.get("brier", 1.0),
|
| 561 |
+
"roi": final_best.fitness.get("roi", 0.0),
|
| 562 |
+
"sharpe": final_best.fitness.get("sharpe", 0.0),
|
| 563 |
+
"calibration": final_best.fitness.get("calibration", 1.0),
|
| 564 |
+
"composite": final_best.fitness.get("composite", 0.0),
|
| 565 |
+
"best_ever_brier": best_ever_brier,
|
| 566 |
+
"best_ever_composite": best_ever_composite,
|
| 567 |
+
"n_generations": n_gens,
|
| 568 |
+
"pop_size": pop_size,
|
| 569 |
+
"mutation_rate": mutation_rate,
|
| 570 |
+
"crossover_rate": crossover_rate,
|
| 571 |
+
"target_features": target_features,
|
| 572 |
+
"history": history,
|
| 573 |
+
"final_model_type": final_best.hyperparams.get("model_type", "unknown"),
|
| 574 |
+
"final_n_features": final_best.n_features,
|
| 575 |
+
}
|
| 576 |
+
|
| 577 |
+
|
| 578 |
+
# ═══════════════════════════════════════════════════════
|
| 579 |
+
# HELPERS
|
| 580 |
+
# ═══════════════════════════════════════════════════════
|
| 581 |
+
|
| 582 |
+
def _load_best_individual(n_features: int) -> Optional[Individual]:
|
| 583 |
+
"""Load the best individual from S10's saved state (read-only, never modifies)."""
|
| 584 |
+
state_file = STATE_DIR / "population.json"
|
| 585 |
+
if not state_file.exists():
|
| 586 |
+
return None
|
| 587 |
+
try:
|
| 588 |
+
st = json.loads(state_file.read_text())
|
| 589 |
+
if not st.get("best_ever"):
|
| 590 |
+
return None
|
| 591 |
+
be = st["best_ever"]
|
| 592 |
+
ind = Individual.__new__(Individual)
|
| 593 |
+
ind.features = be["features"]
|
| 594 |
+
ind.hyperparams = be["hyperparams"]
|
| 595 |
+
ind.fitness = be["fitness"]
|
| 596 |
+
ind.generation = be.get("generation", 0)
|
| 597 |
+
ind.n_features = sum(ind.features)
|
| 598 |
+
|
| 599 |
+
# Resize if feature count changed
|
| 600 |
+
if len(ind.features) < n_features:
|
| 601 |
+
ind.features.extend([0] * (n_features - len(ind.features)))
|
| 602 |
+
elif len(ind.features) > n_features:
|
| 603 |
+
ind.features = ind.features[:n_features]
|
| 604 |
+
ind.n_features = sum(ind.features)
|
| 605 |
+
|
| 606 |
+
return ind
|
| 607 |
+
except Exception as e:
|
| 608 |
+
log(f"[EXPERIMENT] Failed to load best individual: {e}", "WARN")
|
| 609 |
+
return None
|
| 610 |
+
|
| 611 |
+
|
| 612 |
+
def _compute_accuracy(ind: Individual, X: np.ndarray, y: np.ndarray) -> float:
|
| 613 |
+
"""Compute accuracy for an individual (separate from evaluate's fitness)."""
|
| 614 |
+
try:
|
| 615 |
+
from sklearn.model_selection import TimeSeriesSplit
|
| 616 |
+
from sklearn.metrics import accuracy_score
|
| 617 |
+
from sklearn.base import clone
|
| 618 |
+
from sklearn.calibration import CalibratedClassifierCV
|
| 619 |
+
|
| 620 |
+
selected = ind.selected_indices()
|
| 621 |
+
if len(selected) < 15:
|
| 622 |
+
return 0.0
|
| 623 |
+
|
| 624 |
+
X_sub = np.nan_to_num(X[:, selected], nan=0.0, posinf=1e6, neginf=-1e6)
|
| 625 |
+
X_sub, _ = _prune_correlated_features(X_sub, threshold=0.95)
|
| 626 |
+
|
| 627 |
+
hp = ind.hyperparams
|
| 628 |
+
model = _build(hp)
|
| 629 |
+
if model is None:
|
| 630 |
+
return 0.0
|
| 631 |
+
|
| 632 |
+
tscv = TimeSeriesSplit(n_splits=3)
|
| 633 |
+
accs = []
|
| 634 |
+
for ti, vi in tscv.split(X_sub):
|
| 635 |
+
try:
|
| 636 |
+
m = clone(model)
|
| 637 |
+
if hp.get("calibration", "none") != "none":
|
| 638 |
+
m = CalibratedClassifierCV(m, method=hp["calibration"], cv=2)
|
| 639 |
+
m.fit(X_sub[ti], y[ti])
|
| 640 |
+
preds = m.predict(X_sub[vi])
|
| 641 |
+
accs.append(accuracy_score(y[vi], preds))
|
| 642 |
+
except Exception:
|
| 643 |
+
pass
|
| 644 |
+
return round(float(np.mean(accs)), 4) if accs else 0.0
|
| 645 |
+
except Exception:
|
| 646 |
+
return 0.0
|
| 647 |
+
|
| 648 |
+
|
| 649 |
+
# ═══════════════════════════════════════════════════════
|
| 650 |
+
# EXPERIMENT EXECUTION ENGINE
|
| 651 |
+
# ═══════════════════════════════════════════════════════
|
| 652 |
+
|
| 653 |
+
# Experiment state (for status API)
|
| 654 |
+
_current_experiment = None
|
| 655 |
+
_queue_depth = 0
|
| 656 |
+
_experiments_completed = 0
|
| 657 |
+
_experiments_failed = 0
|
| 658 |
+
_last_result = None
|
| 659 |
+
|
| 660 |
+
|
| 661 |
+
EXECUTORS = {
|
| 662 |
+
"feature_test": run_feature_test,
|
| 663 |
+
"model_test": run_model_test,
|
| 664 |
+
"calibration_test": run_calibration_test,
|
| 665 |
+
"config_change": run_config_change,
|
| 666 |
+
}
|
| 667 |
+
|
| 668 |
+
|
| 669 |
+
def run_experiment(experiment: dict, X: np.ndarray, y: np.ndarray,
|
| 670 |
+
feature_names: list) -> dict:
|
| 671 |
+
"""Route an experiment to the correct executor and return results."""
|
| 672 |
+
global _current_experiment, _experiments_completed, _experiments_failed, _last_result
|
| 673 |
+
|
| 674 |
+
exp_type = experiment["experiment_type"]
|
| 675 |
+
exp_id = experiment["id"]
|
| 676 |
+
_current_experiment = experiment
|
| 677 |
+
|
| 678 |
+
log(f"[EXPERIMENT] === Starting: {experiment['experiment_id']} ===")
|
| 679 |
+
log(f"[EXPERIMENT] Type: {exp_type} | Agent: {experiment['agent_name']} | Priority: {experiment['priority']}")
|
| 680 |
+
log(f"[EXPERIMENT] Description: {experiment['description'][:200]}")
|
| 681 |
+
|
| 682 |
+
executor = EXECUTORS.get(exp_type)
|
| 683 |
+
if not executor:
|
| 684 |
+
error_msg = f"Unknown experiment type: {exp_type}. Valid: {list(EXECUTORS.keys())}"
|
| 685 |
+
fail_experiment(exp_id, error_msg)
|
| 686 |
+
_experiments_failed += 1
|
| 687 |
+
_current_experiment = None
|
| 688 |
+
raise ValueError(error_msg)
|
| 689 |
+
|
| 690 |
+
# Claim it
|
| 691 |
+
if not claim_experiment(exp_id):
|
| 692 |
+
_current_experiment = None
|
| 693 |
+
raise RuntimeError(f"Experiment {exp_id} already claimed by another runner")
|
| 694 |
+
|
| 695 |
+
start_time = time.time()
|
| 696 |
+
try:
|
| 697 |
+
results = executor(experiment, X, y, feature_names)
|
| 698 |
+
elapsed = time.time() - start_time
|
| 699 |
+
|
| 700 |
+
brier = float(results.get("brier", 1.0))
|
| 701 |
+
accuracy = float(_compute_accuracy(
|
| 702 |
+
_make_individual(X.shape[1], experiment["params"]), X, y
|
| 703 |
+
)) if exp_type != "config_change" else float(results.get("accuracy", 0.0))
|
| 704 |
+
|
| 705 |
+
# Compute log_loss from brier (approximate — actual log_loss needs probabilities)
|
| 706 |
+
log_loss_val = float(results.get("log_loss", 0.0))
|
| 707 |
+
|
| 708 |
+
results["elapsed_seconds"] = round(elapsed, 1)
|
| 709 |
+
results["games_evaluated"] = min(X.shape[0], MAX_EVAL_GAMES)
|
| 710 |
+
results["feature_candidates"] = X.shape[1]
|
| 711 |
+
results["experiment_id"] = experiment["experiment_id"]
|
| 712 |
+
results["agent_name"] = experiment["agent_name"]
|
| 713 |
+
|
| 714 |
+
# Compare with baseline
|
| 715 |
+
baseline = experiment.get("baseline_brier")
|
| 716 |
+
if baseline:
|
| 717 |
+
results["improvement"] = round(baseline - brier, 5)
|
| 718 |
+
results["improved"] = brier < baseline
|
| 719 |
+
|
| 720 |
+
complete_experiment(exp_id, brier, accuracy, log_loss_val, results)
|
| 721 |
+
_experiments_completed += 1
|
| 722 |
+
_last_result = results
|
| 723 |
+
|
| 724 |
+
log(f"[EXPERIMENT] === Completed: {experiment['experiment_id']} ===")
|
| 725 |
+
log(f"[EXPERIMENT] Brier: {brier:.4f} | Elapsed: {elapsed:.1f}s")
|
| 726 |
+
if baseline:
|
| 727 |
+
delta = baseline - brier
|
| 728 |
+
log(f"[EXPERIMENT] vs baseline {baseline:.4f}: {'BETTER' if delta > 0 else 'WORSE'} by {abs(delta):.4f}")
|
| 729 |
+
|
| 730 |
+
return results
|
| 731 |
+
|
| 732 |
+
except Exception as e:
|
| 733 |
+
elapsed = time.time() - start_time
|
| 734 |
+
error_msg = f"{str(e)[:500]}\n{traceback.format_exc()[-1000:]}"
|
| 735 |
+
fail_experiment(exp_id, error_msg)
|
| 736 |
+
_experiments_failed += 1
|
| 737 |
+
log(f"[EXPERIMENT] === FAILED: {experiment['experiment_id']} ({elapsed:.1f}s) ===", "ERROR")
|
| 738 |
+
log(f"[EXPERIMENT] Error: {str(e)[:300]}", "ERROR")
|
| 739 |
+
raise
|
| 740 |
+
finally:
|
| 741 |
+
_current_experiment = None
|
| 742 |
+
|
| 743 |
+
|
| 744 |
+
# ═══════════════════════════════════════════════════════
|
| 745 |
+
# MAIN LOOP (replaces evolution_loop on S11)
|
| 746 |
+
# ═══════════════════════════════════════════════════════
|
| 747 |
+
|
| 748 |
+
def experiment_loop():
|
| 749 |
+
"""Main experiment polling loop — runs in background thread on S11.
|
| 750 |
+
|
| 751 |
+
1. Load game data + build features (same as evolution_loop init)
|
| 752 |
+
2. Poll Supabase every 60s for pending experiments
|
| 753 |
+
3. Execute one at a time, write results back
|
| 754 |
+
4. Never modifies S10's population or state
|
| 755 |
+
"""
|
| 756 |
+
global _queue_depth
|
| 757 |
+
|
| 758 |
+
# Initialize references to app.py functions (must be called after app.py is loaded)
|
| 759 |
+
init_from_app()
|
| 760 |
+
|
| 761 |
+
log("=" * 60)
|
| 762 |
+
log("S11 EXPERIMENT RUNNER — STARTING")
|
| 763 |
+
log("=" * 60)
|
| 764 |
+
|
| 765 |
+
# ── Phase 1: Load data (same as evolution_loop) ──
|
| 766 |
+
live["status"] = "EXPERIMENT: LOADING DATA"
|
| 767 |
+
pull_seasons()
|
| 768 |
+
games = load_all_games()
|
| 769 |
+
live["games"] = len(games)
|
| 770 |
+
log(f"[EXPERIMENT] Games loaded: {len(games)}")
|
| 771 |
+
|
| 772 |
+
if len(games) < 500:
|
| 773 |
+
log("[EXPERIMENT] NOT ENOUGH GAMES — aborting", "ERROR")
|
| 774 |
+
live["status"] = "EXPERIMENT: ERROR (no data)"
|
| 775 |
+
return
|
| 776 |
+
|
| 777 |
+
# ── Phase 2: Build features ──
|
| 778 |
+
live["status"] = "EXPERIMENT: BUILDING FEATURES"
|
| 779 |
+
X, y, feature_names = build_features(games)
|
| 780 |
+
log(f"[EXPERIMENT] Raw feature matrix: {X.shape}")
|
| 781 |
+
|
| 782 |
+
# Remove zero-variance features (same filter as evolution_loop)
|
| 783 |
+
variances = np.var(X, axis=0)
|
| 784 |
+
valid_mask = variances > 1e-10
|
| 785 |
+
n_removed = int((~valid_mask).sum())
|
| 786 |
+
if n_removed > 0:
|
| 787 |
+
X = X[:, valid_mask]
|
| 788 |
+
feature_names = [f for f, v in zip(feature_names, valid_mask) if v]
|
| 789 |
+
log(f"[EXPERIMENT] Noise filter: removed {n_removed} zero-variance features")
|
| 790 |
+
|
| 791 |
+
n_feat = X.shape[1]
|
| 792 |
+
live["feature_candidates"] = n_feat
|
| 793 |
+
log(f"[EXPERIMENT] Clean feature matrix: {X.shape} ({n_feat} usable features)")
|
| 794 |
+
|
| 795 |
+
# Cap games for OOM protection
|
| 796 |
+
if X.shape[0] > MAX_EVAL_GAMES:
|
| 797 |
+
log(f"[EXPERIMENT] Capping to {MAX_EVAL_GAMES} most recent games (OOM protection)")
|
| 798 |
+
X = X[-MAX_EVAL_GAMES:]
|
| 799 |
+
y = y[-MAX_EVAL_GAMES:]
|
| 800 |
+
|
| 801 |
+
# ── Phase 3: Poll loop ──
|
| 802 |
+
live["status"] = "EXPERIMENT: READY (polling)"
|
| 803 |
+
log(f"[EXPERIMENT] Ready — polling every {POLL_INTERVAL}s for experiments")
|
| 804 |
+
consecutive_errors = 0
|
| 805 |
+
|
| 806 |
+
while True:
|
| 807 |
+
try:
|
| 808 |
+
experiment = fetch_next_experiment()
|
| 809 |
+
|
| 810 |
+
if experiment:
|
| 811 |
+
_queue_depth = _count_pending()
|
| 812 |
+
live["status"] = f"EXPERIMENT: RUNNING ({experiment['experiment_type']})"
|
| 813 |
+
log(f"[EXPERIMENT] Found experiment: {experiment['experiment_id']} "
|
| 814 |
+
f"(type={experiment['experiment_type']}, queue={_queue_depth})")
|
| 815 |
+
|
| 816 |
+
try:
|
| 817 |
+
run_experiment(experiment, X, y, feature_names)
|
| 818 |
+
except Exception as e:
|
| 819 |
+
log(f"[EXPERIMENT] Experiment failed: {e}", "ERROR")
|
| 820 |
+
|
| 821 |
+
live["status"] = "EXPERIMENT: READY (polling)"
|
| 822 |
+
consecutive_errors = 0
|
| 823 |
+
|
| 824 |
+
# Refresh data every 10 experiments
|
| 825 |
+
if (_experiments_completed + _experiments_failed) % 10 == 0:
|
| 826 |
+
try:
|
| 827 |
+
log("[EXPERIMENT] Refreshing game data...")
|
| 828 |
+
new_games = load_all_games()
|
| 829 |
+
if len(new_games) > len(games):
|
| 830 |
+
games = new_games
|
| 831 |
+
X_new, y_new, fn_new = build_features(games)
|
| 832 |
+
variances = np.var(X_new, axis=0)
|
| 833 |
+
valid_mask = variances > 1e-10
|
| 834 |
+
X_new = X_new[:, valid_mask]
|
| 835 |
+
fn_new = [f for f, v in zip(fn_new, valid_mask) if v]
|
| 836 |
+
if X_new.shape[0] > MAX_EVAL_GAMES:
|
| 837 |
+
X_new = X_new[-MAX_EVAL_GAMES:]
|
| 838 |
+
y_new = y_new[-MAX_EVAL_GAMES:]
|
| 839 |
+
X, y, feature_names = X_new, y_new, fn_new
|
| 840 |
+
n_feat = X.shape[1]
|
| 841 |
+
log(f"[EXPERIMENT] Data refreshed: {X.shape}")
|
| 842 |
+
except Exception as e:
|
| 843 |
+
log(f"[EXPERIMENT] Data refresh failed (continuing with old): {e}", "WARN")
|
| 844 |
+
|
| 845 |
+
else:
|
| 846 |
+
# No experiments pending — sleep
|
| 847 |
+
time.sleep(POLL_INTERVAL)
|
| 848 |
+
|
| 849 |
+
except Exception as e:
|
| 850 |
+
consecutive_errors += 1
|
| 851 |
+
log(f"[EXPERIMENT] Poll error #{consecutive_errors}: {e}", "ERROR")
|
| 852 |
+
|
| 853 |
+
if consecutive_errors >= 5:
|
| 854 |
+
log("[EXPERIMENT] 5 consecutive errors — reconnecting to Supabase", "WARN")
|
| 855 |
+
_reconnect_pg()
|
| 856 |
+
consecutive_errors = 0
|
| 857 |
+
|
| 858 |
+
time.sleep(POLL_INTERVAL * 2) # Back off on errors
|
| 859 |
+
|
| 860 |
+
|
| 861 |
+
def _count_pending() -> int:
|
| 862 |
+
"""Count pending experiments in queue."""
|
| 863 |
+
rows = _exec_sql("""
|
| 864 |
+
SELECT COUNT(*) FROM public.nba_experiments
|
| 865 |
+
WHERE status = 'pending'
|
| 866 |
+
AND (target_space IS NULL OR target_space = 'S11' OR target_space = 'any')
|
| 867 |
+
""")
|
| 868 |
+
if rows and rows is not True and len(rows) > 0:
|
| 869 |
+
return rows[0][0]
|
| 870 |
+
return 0
|
| 871 |
+
|
| 872 |
+
|
| 873 |
+
# ═══════════════════════════════════════════════════════
|
| 874 |
+
# FASTAPI ENDPOINTS (added to control_api)
|
| 875 |
+
# ═══════════════════════════════════════════════════════
|
| 876 |
+
|
| 877 |
+
def register_experiment_endpoints(api):
|
| 878 |
+
"""Register experiment-related FastAPI endpoints on the control_api."""
|
| 879 |
+
from fastapi import Request
|
| 880 |
+
from fastapi.responses import JSONResponse
|
| 881 |
+
|
| 882 |
+
@api.get("/api/experiment/status")
|
| 883 |
+
async def experiment_status():
|
| 884 |
+
"""Current experiment runner status."""
|
| 885 |
+
return JSONResponse({
|
| 886 |
+
"mode": "experiment_runner",
|
| 887 |
+
"status": live.get("status", "unknown"),
|
| 888 |
+
"current_experiment": {
|
| 889 |
+
"experiment_id": _current_experiment["experiment_id"],
|
| 890 |
+
"type": _current_experiment["experiment_type"],
|
| 891 |
+
"agent": _current_experiment["agent_name"],
|
| 892 |
+
"description": _current_experiment["description"][:200],
|
| 893 |
+
} if _current_experiment else None,
|
| 894 |
+
"queue_depth": _queue_depth,
|
| 895 |
+
"experiments_completed": _experiments_completed,
|
| 896 |
+
"experiments_failed": _experiments_failed,
|
| 897 |
+
"last_result": {
|
| 898 |
+
"experiment_id": _last_result.get("experiment_id"),
|
| 899 |
+
"brier": _last_result.get("brier"),
|
| 900 |
+
"improvement": _last_result.get("improvement"),
|
| 901 |
+
"elapsed_seconds": _last_result.get("elapsed_seconds"),
|
| 902 |
+
} if _last_result else None,
|
| 903 |
+
"timestamp": datetime.now(timezone.utc).isoformat(),
|
| 904 |
+
})
|
| 905 |
+
|
| 906 |
+
@api.post("/api/experiment/run")
|
| 907 |
+
async def experiment_run_direct(request: Request):
|
| 908 |
+
"""Submit and immediately run an experiment (bypasses queue).
|
| 909 |
+
|
| 910 |
+
Body: {
|
| 911 |
+
"experiment_type": "feature_test|model_test|calibration_test|config_change",
|
| 912 |
+
"description": "...",
|
| 913 |
+
"params": { ... },
|
| 914 |
+
"agent_name": "direct_api"
|
| 915 |
+
}
|
| 916 |
+
"""
|
| 917 |
+
try:
|
| 918 |
+
body = await request.json()
|
| 919 |
+
|
| 920 |
+
exp_type = body.get("experiment_type")
|
| 921 |
+
if exp_type not in EXECUTORS:
|
| 922 |
+
return JSONResponse(
|
| 923 |
+
{"error": f"Invalid experiment_type. Valid: {list(EXECUTORS.keys())}"},
|
| 924 |
+
status_code=400
|
| 925 |
+
)
|
| 926 |
+
|
| 927 |
+
# Write to Supabase first (for tracking), then execute
|
| 928 |
+
exp_id_str = f"direct-{int(time.time())}"
|
| 929 |
+
_exec_sql("""
|
| 930 |
+
INSERT INTO public.nba_experiments
|
| 931 |
+
(experiment_id, agent_name, experiment_type, description, hypothesis,
|
| 932 |
+
params, priority, status, target_space, feature_engine_version)
|
| 933 |
+
VALUES (%s, %s, %s, %s, %s, %s, %s, 'pending', 'S11', %s)
|
| 934 |
+
RETURNING id
|
| 935 |
+
""", (
|
| 936 |
+
exp_id_str,
|
| 937 |
+
body.get("agent_name", "direct_api"),
|
| 938 |
+
exp_type,
|
| 939 |
+
body.get("description", "Direct API submission"),
|
| 940 |
+
body.get("hypothesis", ""),
|
| 941 |
+
json.dumps(body.get("params", {})),
|
| 942 |
+
body.get("priority", 10),
|
| 943 |
+
"v3.0-35cat-6000feat",
|
| 944 |
+
))
|
| 945 |
+
|
| 946 |
+
# Fetch the just-inserted experiment
|
| 947 |
+
rows = _exec_sql("""
|
| 948 |
+
SELECT id, experiment_id, agent_name, experiment_type, description,
|
| 949 |
+
hypothesis, params, priority, status, target_space,
|
| 950 |
+
baseline_brier, created_at
|
| 951 |
+
FROM public.nba_experiments
|
| 952 |
+
WHERE experiment_id = %s
|
| 953 |
+
ORDER BY id DESC LIMIT 1
|
| 954 |
+
""", (exp_id_str,))
|
| 955 |
+
|
| 956 |
+
if not rows or rows is True:
|
| 957 |
+
return JSONResponse({"error": "Failed to insert experiment"}, status_code=500)
|
| 958 |
+
|
| 959 |
+
row = rows[0]
|
| 960 |
+
experiment = {
|
| 961 |
+
"id": row[0], "experiment_id": row[1], "agent_name": row[2],
|
| 962 |
+
"experiment_type": row[3], "description": row[4], "hypothesis": row[5],
|
| 963 |
+
"params": row[6] if isinstance(row[6], dict) else json.loads(row[6]) if row[6] else {},
|
| 964 |
+
"priority": row[7], "status": row[8], "target_space": row[9],
|
| 965 |
+
"baseline_brier": row[10], "created_at": str(row[11]) if row[11] else None,
|
| 966 |
+
}
|
| 967 |
+
|
| 968 |
+
# Note: This blocks the request until evaluation completes (can be long).
|
| 969 |
+
# For non-blocking, use /api/experiment/submit instead.
|
| 970 |
+
return JSONResponse({
|
| 971 |
+
"status": "queued_for_poll",
|
| 972 |
+
"experiment_id": exp_id_str,
|
| 973 |
+
"message": "Experiment inserted into queue. S11 will pick it up on next poll cycle.",
|
| 974 |
+
})
|
| 975 |
+
|
| 976 |
+
except Exception as e:
|
| 977 |
+
return JSONResponse({"error": str(e)[:500]}, status_code=500)
|
| 978 |
+
|
| 979 |
+
@api.post("/api/experiment/submit")
|
| 980 |
+
async def experiment_submit(request: Request):
|
| 981 |
+
"""Submit an experiment to the Supabase queue (non-blocking).
|
| 982 |
+
|
| 983 |
+
Body: {
|
| 984 |
+
"experiment_id": "optional-custom-id",
|
| 985 |
+
"agent_name": "eve|crew-research|crew-feature|manual",
|
| 986 |
+
"experiment_type": "feature_test|model_test|calibration_test|config_change",
|
| 987 |
+
"description": "What this experiment tests",
|
| 988 |
+
"hypothesis": "Expected outcome",
|
| 989 |
+
"params": { ... },
|
| 990 |
+
"priority": 5,
|
| 991 |
+
"baseline_brier": 0.2205
|
| 992 |
+
}
|
| 993 |
+
"""
|
| 994 |
+
try:
|
| 995 |
+
body = await request.json()
|
| 996 |
+
|
| 997 |
+
exp_type = body.get("experiment_type")
|
| 998 |
+
if exp_type not in EXECUTORS:
|
| 999 |
+
return JSONResponse(
|
| 1000 |
+
{"error": f"Invalid experiment_type. Valid: {list(EXECUTORS.keys())}"},
|
| 1001 |
+
status_code=400
|
| 1002 |
+
)
|
| 1003 |
+
|
| 1004 |
+
if not body.get("description"):
|
| 1005 |
+
return JSONResponse({"error": "description is required"}, status_code=400)
|
| 1006 |
+
|
| 1007 |
+
exp_id_str = body.get("experiment_id", f"submit-{int(time.time())}")
|
| 1008 |
+
|
| 1009 |
+
result = _exec_sql("""
|
| 1010 |
+
INSERT INTO public.nba_experiments
|
| 1011 |
+
(experiment_id, agent_name, experiment_type, description, hypothesis,
|
| 1012 |
+
params, priority, status, target_space, baseline_brier, feature_engine_version)
|
| 1013 |
+
VALUES (%s, %s, %s, %s, %s, %s, %s, 'pending', %s, %s, %s)
|
| 1014 |
+
RETURNING id
|
| 1015 |
+
""", (
|
| 1016 |
+
exp_id_str,
|
| 1017 |
+
body.get("agent_name", "unknown"),
|
| 1018 |
+
exp_type,
|
| 1019 |
+
body["description"],
|
| 1020 |
+
body.get("hypothesis", ""),
|
| 1021 |
+
json.dumps(body.get("params", {})),
|
| 1022 |
+
body.get("priority", 5),
|
| 1023 |
+
body.get("target_space", "S11"),
|
| 1024 |
+
body.get("baseline_brier"),
|
| 1025 |
+
body.get("feature_engine_version", "v3.0-35cat-6000feat"),
|
| 1026 |
+
))
|
| 1027 |
+
|
| 1028 |
+
if result is None:
|
| 1029 |
+
return JSONResponse({"error": "Failed to insert — Supabase error"}, status_code=500)
|
| 1030 |
+
|
| 1031 |
+
db_id = result[0][0] if result and result is not True else None
|
| 1032 |
+
|
| 1033 |
+
return JSONResponse({
|
| 1034 |
+
"status": "queued",
|
| 1035 |
+
"id": db_id,
|
| 1036 |
+
"experiment_id": exp_id_str,
|
| 1037 |
+
"message": f"Experiment queued. S11 polls every {POLL_INTERVAL}s.",
|
| 1038 |
+
})
|
| 1039 |
+
|
| 1040 |
+
except Exception as e:
|
| 1041 |
+
return JSONResponse({"error": str(e)[:500]}, status_code=500)
|
| 1042 |
+
|
| 1043 |
+
@api.get("/api/experiment/results")
|
| 1044 |
+
async def experiment_results():
|
| 1045 |
+
"""Fetch recent experiment results from Supabase."""
|
| 1046 |
+
rows = _exec_sql("""
|
| 1047 |
+
SELECT experiment_id, agent_name, experiment_type, status,
|
| 1048 |
+
result_brier, result_accuracy, result_details,
|
| 1049 |
+
created_at, completed_at
|
| 1050 |
+
FROM public.nba_experiments
|
| 1051 |
+
ORDER BY id DESC
|
| 1052 |
+
LIMIT 20
|
| 1053 |
+
""")
|
| 1054 |
+
if not rows or rows is True:
|
| 1055 |
+
return JSONResponse({"experiments": [], "count": 0})
|
| 1056 |
+
|
| 1057 |
+
experiments = []
|
| 1058 |
+
for row in rows:
|
| 1059 |
+
experiments.append({
|
| 1060 |
+
"experiment_id": row[0],
|
| 1061 |
+
"agent_name": row[1],
|
| 1062 |
+
"experiment_type": row[2],
|
| 1063 |
+
"status": row[3],
|
| 1064 |
+
"result_brier": row[4],
|
| 1065 |
+
"result_accuracy": row[5],
|
| 1066 |
+
"result_details": row[6],
|
| 1067 |
+
"created_at": str(row[7]) if row[7] else None,
|
| 1068 |
+
"completed_at": str(row[8]) if row[8] else None,
|
| 1069 |
+
})
|
| 1070 |
+
|
| 1071 |
+
return JSONResponse({"experiments": experiments, "count": len(experiments)})
|
| 1072 |
+
|
| 1073 |
+
log("[EXPERIMENT] FastAPI endpoints registered: /api/experiment/{status,run,submit,results}")
|
features/__init__.py
ADDED
|
File without changes
|
features/engine.py
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
models/__init__.py
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
NBA Quant AI — Neural Network Models
|
| 3 |
+
=====================================
|
| 4 |
+
SOTA 2025-2026 neural architectures for NBA game prediction.
|
| 5 |
+
|
| 6 |
+
All models conform to the same interface:
|
| 7 |
+
- fit(X_train, y_train, X_val, y_val)
|
| 8 |
+
- predict_proba(X)
|
| 9 |
+
- get_params()
|
| 10 |
+
- save(path) / load(path)
|
| 11 |
+
|
| 12 |
+
Runs on HF Spaces (16 GB RAM, CPU-only PyTorch).
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from .neural_models import (
|
| 16 |
+
LSTMSequenceModel,
|
| 17 |
+
TransformerAttentionModel,
|
| 18 |
+
TabNetModel,
|
| 19 |
+
FTTransformerModel,
|
| 20 |
+
DeepEnsemble,
|
| 21 |
+
ConformalPredictionWrapper,
|
| 22 |
+
AutoGluonEnsemble,
|
| 23 |
+
NEURAL_MODEL_REGISTRY,
|
| 24 |
+
)
|
| 25 |
+
|
| 26 |
+
__all__ = [
|
| 27 |
+
"LSTMSequenceModel",
|
| 28 |
+
"TransformerAttentionModel",
|
| 29 |
+
"TabNetModel",
|
| 30 |
+
"FTTransformerModel",
|
| 31 |
+
"DeepEnsemble",
|
| 32 |
+
"ConformalPredictionWrapper",
|
| 33 |
+
"AutoGluonEnsemble",
|
| 34 |
+
"NEURAL_MODEL_REGISTRY",
|
| 35 |
+
]
|
models/neural_models.py
ADDED
|
@@ -0,0 +1,1598 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
NBA Quant AI — Neural Network Models (2025-2026 SOTA)
|
| 4 |
+
======================================================
|
| 5 |
+
Real, production-grade neural architectures for NBA game prediction.
|
| 6 |
+
|
| 7 |
+
Models implemented:
|
| 8 |
+
1. LSTMSequenceModel — Bidirectional LSTM over last N games
|
| 9 |
+
2. TransformerAttentionModel — Self-attention over game history
|
| 10 |
+
3. TabNetModel — Attention-based tabular learning (Arik & Pfister 2021)
|
| 11 |
+
4. FTTransformerModel — Feature Tokenizer + Transformer (Gorishniy et al. 2021)
|
| 12 |
+
5. DeepEnsemble — N independent nets, averaged predictions
|
| 13 |
+
6. ConformalPredictionWrapper — Calibrated prediction intervals (any base model)
|
| 14 |
+
7. AutoGluonEnsemble — Auto-stacking over hundreds of configs
|
| 15 |
+
|
| 16 |
+
All models:
|
| 17 |
+
- Handle NaN gracefully (median imputation)
|
| 18 |
+
- Work with 6000+ features
|
| 19 |
+
- Use early stopping
|
| 20 |
+
- CPU-only PyTorch (no CUDA needed)
|
| 21 |
+
- Fit in 16 GB RAM (HF Spaces free tier)
|
| 22 |
+
|
| 23 |
+
THIS RUNS ON HF SPACES ONLY — NOT ON VM.
|
| 24 |
+
"""
|
| 25 |
+
|
| 26 |
+
from __future__ import annotations
|
| 27 |
+
|
| 28 |
+
import copy
|
| 29 |
+
import json
|
| 30 |
+
import math
|
| 31 |
+
import os
|
| 32 |
+
import pickle
|
| 33 |
+
import warnings
|
| 34 |
+
from abc import ABC, abstractmethod
|
| 35 |
+
from pathlib import Path
|
| 36 |
+
from typing import Any, Dict, List, Optional, Tuple, Union
|
| 37 |
+
|
| 38 |
+
import numpy as np
|
| 39 |
+
from sklearn.model_selection import train_test_split
|
| 40 |
+
from sklearn.preprocessing import StandardScaler
|
| 41 |
+
|
| 42 |
+
warnings.filterwarnings("ignore", category=UserWarning)
|
| 43 |
+
|
| 44 |
+
# ---------------------------------------------------------------------------
|
| 45 |
+
# Lazy imports — heavy libraries loaded only when a model is instantiated
|
| 46 |
+
# ---------------------------------------------------------------------------
|
| 47 |
+
|
| 48 |
+
def _import_torch():
|
| 49 |
+
"""Import torch lazily to avoid startup cost."""
|
| 50 |
+
import torch
|
| 51 |
+
import torch.nn as nn
|
| 52 |
+
import torch.optim as optim
|
| 53 |
+
from torch.utils.data import DataLoader, TensorDataset
|
| 54 |
+
return torch, nn, optim, DataLoader, TensorDataset
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
# ---------------------------------------------------------------------------
|
| 58 |
+
# Base class — common interface for all models
|
| 59 |
+
# ---------------------------------------------------------------------------
|
| 60 |
+
|
| 61 |
+
class BaseNBAModel(ABC):
|
| 62 |
+
"""Abstract base for all NBA prediction models."""
|
| 63 |
+
|
| 64 |
+
def __init__(self, **params):
|
| 65 |
+
self.params = params
|
| 66 |
+
self._scaler: Optional[StandardScaler] = None
|
| 67 |
+
self._feature_medians: Optional[np.ndarray] = None
|
| 68 |
+
self._is_fitted = False
|
| 69 |
+
|
| 70 |
+
# --- public interface ---------------------------------------------------
|
| 71 |
+
|
| 72 |
+
@abstractmethod
|
| 73 |
+
def fit(
|
| 74 |
+
self,
|
| 75 |
+
X_train: np.ndarray,
|
| 76 |
+
y_train: np.ndarray,
|
| 77 |
+
X_val: Optional[np.ndarray] = None,
|
| 78 |
+
y_val: Optional[np.ndarray] = None,
|
| 79 |
+
) -> "BaseNBAModel":
|
| 80 |
+
"""Train the model. Returns self."""
|
| 81 |
+
...
|
| 82 |
+
|
| 83 |
+
@abstractmethod
|
| 84 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 85 |
+
"""Return P(home_win) for each row — shape (n,)."""
|
| 86 |
+
...
|
| 87 |
+
|
| 88 |
+
def get_params(self) -> Dict[str, Any]:
|
| 89 |
+
"""Return hyperparameter dict (JSON-serialisable)."""
|
| 90 |
+
return {k: v for k, v in self.params.items() if _is_jsonable(v)}
|
| 91 |
+
|
| 92 |
+
def save(self, path: Union[str, Path]) -> None:
|
| 93 |
+
"""Persist to disk."""
|
| 94 |
+
path = Path(path)
|
| 95 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 96 |
+
with open(path, "wb") as f:
|
| 97 |
+
pickle.dump(self, f, protocol=pickle.HIGHEST_PROTOCOL)
|
| 98 |
+
|
| 99 |
+
@classmethod
|
| 100 |
+
def load(cls, path: Union[str, Path]) -> "BaseNBAModel":
|
| 101 |
+
"""Load from disk."""
|
| 102 |
+
with open(path, "rb") as f:
|
| 103 |
+
obj = pickle.load(f)
|
| 104 |
+
return obj
|
| 105 |
+
|
| 106 |
+
# --- NaN handling & scaling --------------------------------------------
|
| 107 |
+
|
| 108 |
+
def _impute(self, X: np.ndarray, fit: bool = False) -> np.ndarray:
|
| 109 |
+
"""Replace NaN/Inf with column medians. If *fit*, compute medians first."""
|
| 110 |
+
X = np.array(X, dtype=np.float32)
|
| 111 |
+
X = np.where(np.isfinite(X), X, np.nan)
|
| 112 |
+
if fit:
|
| 113 |
+
self._feature_medians = np.nanmedian(X, axis=0)
|
| 114 |
+
self._feature_medians = np.where(
|
| 115 |
+
np.isfinite(self._feature_medians), self._feature_medians, 0.0
|
| 116 |
+
)
|
| 117 |
+
medians = self._feature_medians if self._feature_medians is not None else np.zeros(X.shape[1])
|
| 118 |
+
inds = np.where(np.isnan(X))
|
| 119 |
+
X[inds] = np.take(medians, inds[1])
|
| 120 |
+
return X
|
| 121 |
+
|
| 122 |
+
def _scale(self, X: np.ndarray, fit: bool = False) -> np.ndarray:
|
| 123 |
+
"""Standard-scale features."""
|
| 124 |
+
if fit:
|
| 125 |
+
self._scaler = StandardScaler()
|
| 126 |
+
return self._scaler.fit_transform(X).astype(np.float32)
|
| 127 |
+
if self._scaler is not None:
|
| 128 |
+
return self._scaler.transform(X).astype(np.float32)
|
| 129 |
+
return X.astype(np.float32)
|
| 130 |
+
|
| 131 |
+
def _prepare(self, X: np.ndarray, fit: bool = False) -> np.ndarray:
|
| 132 |
+
"""Impute + scale."""
|
| 133 |
+
X = self._impute(X, fit=fit)
|
| 134 |
+
X = self._scale(X, fit=fit)
|
| 135 |
+
return X
|
| 136 |
+
|
| 137 |
+
def _auto_val_split(
|
| 138 |
+
self,
|
| 139 |
+
X: np.ndarray,
|
| 140 |
+
y: np.ndarray,
|
| 141 |
+
X_val: Optional[np.ndarray],
|
| 142 |
+
y_val: Optional[np.ndarray],
|
| 143 |
+
val_frac: float = 0.15,
|
| 144 |
+
) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
|
| 145 |
+
"""If no validation set provided, carve one from the tail (time-ordered)."""
|
| 146 |
+
if X_val is not None and y_val is not None:
|
| 147 |
+
return X, y, X_val, y_val
|
| 148 |
+
split = int(len(X) * (1 - val_frac))
|
| 149 |
+
return X[:split], y[:split], X[split:], y[split:]
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
# ===========================================================================
|
| 153 |
+
# 1. LSTM Game Sequence Model
|
| 154 |
+
# ===========================================================================
|
| 155 |
+
|
| 156 |
+
class LSTMSequenceModel(BaseNBAModel):
|
| 157 |
+
"""
|
| 158 |
+
Bidirectional LSTM over the last *seq_len* games of features per team.
|
| 159 |
+
|
| 160 |
+
Input shape: (batch, seq_len, n_features)
|
| 161 |
+
Architecture: BiLSTM(128) -> BiLSTM(64) -> Dense(32) -> Sigmoid
|
| 162 |
+
|
| 163 |
+
For flat input (n_samples, n_features), the model internally reshapes
|
| 164 |
+
using a sliding window of *seq_len* rows, treating consecutive games as
|
| 165 |
+
the sequence dimension. For true per-team sequences, pass 3-D arrays
|
| 166 |
+
directly.
|
| 167 |
+
"""
|
| 168 |
+
|
| 169 |
+
def __init__(
|
| 170 |
+
self,
|
| 171 |
+
seq_len: int = 10,
|
| 172 |
+
hidden1: int = 128,
|
| 173 |
+
hidden2: int = 64,
|
| 174 |
+
dense_dim: int = 32,
|
| 175 |
+
dropout: float = 0.3,
|
| 176 |
+
lr: float = 1e-3,
|
| 177 |
+
weight_decay: float = 1e-5,
|
| 178 |
+
batch_size: int = 256,
|
| 179 |
+
epochs: int = 120,
|
| 180 |
+
patience: int = 15,
|
| 181 |
+
**kw,
|
| 182 |
+
):
|
| 183 |
+
super().__init__(
|
| 184 |
+
seq_len=seq_len, hidden1=hidden1, hidden2=hidden2,
|
| 185 |
+
dense_dim=dense_dim, dropout=dropout, lr=lr,
|
| 186 |
+
weight_decay=weight_decay, batch_size=batch_size,
|
| 187 |
+
epochs=epochs, patience=patience, **kw,
|
| 188 |
+
)
|
| 189 |
+
self.seq_len = seq_len
|
| 190 |
+
self.hidden1 = hidden1
|
| 191 |
+
self.hidden2 = hidden2
|
| 192 |
+
self.dense_dim = dense_dim
|
| 193 |
+
self.dropout = dropout
|
| 194 |
+
self.lr = lr
|
| 195 |
+
self.weight_decay = weight_decay
|
| 196 |
+
self.batch_size = batch_size
|
| 197 |
+
self.epochs = epochs
|
| 198 |
+
self.patience = patience
|
| 199 |
+
self._net = None
|
| 200 |
+
|
| 201 |
+
# --- PyTorch module (defined inside method to keep torch lazy) ----------
|
| 202 |
+
|
| 203 |
+
@staticmethod
|
| 204 |
+
def _build_net(n_features: int, cfg: dict):
|
| 205 |
+
torch, nn, _, _, _ = _import_torch()
|
| 206 |
+
|
| 207 |
+
class BiLSTMNet(nn.Module):
|
| 208 |
+
def __init__(self):
|
| 209 |
+
super().__init__()
|
| 210 |
+
self.lstm1 = nn.LSTM(
|
| 211 |
+
input_size=n_features,
|
| 212 |
+
hidden_size=cfg["hidden1"],
|
| 213 |
+
batch_first=True,
|
| 214 |
+
bidirectional=True,
|
| 215 |
+
dropout=cfg["dropout"] if cfg["hidden2"] else 0,
|
| 216 |
+
)
|
| 217 |
+
self.lstm2 = nn.LSTM(
|
| 218 |
+
input_size=cfg["hidden1"] * 2, # bidirectional doubles
|
| 219 |
+
hidden_size=cfg["hidden2"],
|
| 220 |
+
batch_first=True,
|
| 221 |
+
bidirectional=True,
|
| 222 |
+
)
|
| 223 |
+
self.dropout = nn.Dropout(cfg["dropout"])
|
| 224 |
+
self.fc1 = nn.Linear(cfg["hidden2"] * 2, cfg["dense_dim"])
|
| 225 |
+
self.relu = nn.ReLU()
|
| 226 |
+
self.fc2 = nn.Linear(cfg["dense_dim"], 1)
|
| 227 |
+
|
| 228 |
+
def forward(self, x):
|
| 229 |
+
# x: (batch, seq_len, features)
|
| 230 |
+
out, _ = self.lstm1(x)
|
| 231 |
+
out = self.dropout(out)
|
| 232 |
+
out, _ = self.lstm2(out)
|
| 233 |
+
# Take last hidden state
|
| 234 |
+
out = out[:, -1, :]
|
| 235 |
+
out = self.dropout(out)
|
| 236 |
+
out = self.relu(self.fc1(out))
|
| 237 |
+
out = self.dropout(out)
|
| 238 |
+
return torch.sigmoid(self.fc2(out)).squeeze(-1)
|
| 239 |
+
|
| 240 |
+
return BiLSTMNet()
|
| 241 |
+
|
| 242 |
+
# --- Sequence construction from flat arrays ----------------------------
|
| 243 |
+
|
| 244 |
+
def _make_sequences(
|
| 245 |
+
self, X: np.ndarray, y: np.ndarray
|
| 246 |
+
) -> Tuple[np.ndarray, np.ndarray]:
|
| 247 |
+
"""
|
| 248 |
+
Convert flat (n_games, n_features) into (n_sequences, seq_len, n_features).
|
| 249 |
+
Uses a sliding window — game i maps to window [i-seq_len+1 .. i].
|
| 250 |
+
The first seq_len-1 games are dropped (not enough history).
|
| 251 |
+
"""
|
| 252 |
+
if X.ndim == 3:
|
| 253 |
+
return X, y # already sequential
|
| 254 |
+
seqs, labels = [], []
|
| 255 |
+
for i in range(self.seq_len - 1, len(X)):
|
| 256 |
+
seqs.append(X[i - self.seq_len + 1 : i + 1])
|
| 257 |
+
labels.append(y[i])
|
| 258 |
+
return np.array(seqs, dtype=np.float32), np.array(labels, dtype=np.float32)
|
| 259 |
+
|
| 260 |
+
# --- fit / predict -----------------------------------------------------
|
| 261 |
+
|
| 262 |
+
def fit(
|
| 263 |
+
self,
|
| 264 |
+
X_train: np.ndarray,
|
| 265 |
+
y_train: np.ndarray,
|
| 266 |
+
X_val: Optional[np.ndarray] = None,
|
| 267 |
+
y_val: Optional[np.ndarray] = None,
|
| 268 |
+
) -> "LSTMSequenceModel":
|
| 269 |
+
torch, nn, optim, DataLoader, TensorDataset = _import_torch()
|
| 270 |
+
|
| 271 |
+
# Prepare
|
| 272 |
+
X_train = self._prepare(X_train, fit=True)
|
| 273 |
+
X_train, y_train, X_val, y_val = self._auto_val_split(X_train, y_train, X_val, y_val)
|
| 274 |
+
if X_val is not None:
|
| 275 |
+
X_val = self._prepare(X_val)
|
| 276 |
+
|
| 277 |
+
# Build sequences
|
| 278 |
+
X_tr_seq, y_tr_seq = self._make_sequences(X_train, y_train)
|
| 279 |
+
X_va_seq, y_va_seq = self._make_sequences(X_val, y_val)
|
| 280 |
+
|
| 281 |
+
n_features = X_tr_seq.shape[2]
|
| 282 |
+
self._net = self._build_net(n_features, {
|
| 283 |
+
"hidden1": self.hidden1, "hidden2": self.hidden2,
|
| 284 |
+
"dense_dim": self.dense_dim, "dropout": self.dropout,
|
| 285 |
+
})
|
| 286 |
+
|
| 287 |
+
optimizer = optim.AdamW(
|
| 288 |
+
self._net.parameters(), lr=self.lr, weight_decay=self.weight_decay
|
| 289 |
+
)
|
| 290 |
+
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
|
| 291 |
+
optimizer, mode="min", factor=0.5, patience=5, min_lr=1e-6
|
| 292 |
+
)
|
| 293 |
+
criterion = nn.BCELoss()
|
| 294 |
+
|
| 295 |
+
train_ds = TensorDataset(
|
| 296 |
+
torch.from_numpy(X_tr_seq), torch.from_numpy(y_tr_seq)
|
| 297 |
+
)
|
| 298 |
+
train_dl = DataLoader(train_ds, batch_size=self.batch_size, shuffle=True)
|
| 299 |
+
|
| 300 |
+
val_X_t = torch.from_numpy(X_va_seq)
|
| 301 |
+
val_y_t = torch.from_numpy(y_va_seq)
|
| 302 |
+
|
| 303 |
+
best_val_loss = float("inf")
|
| 304 |
+
best_state = None
|
| 305 |
+
wait = 0
|
| 306 |
+
|
| 307 |
+
self._net.train()
|
| 308 |
+
for epoch in range(self.epochs):
|
| 309 |
+
epoch_loss = 0.0
|
| 310 |
+
for xb, yb in train_dl:
|
| 311 |
+
optimizer.zero_grad()
|
| 312 |
+
preds = self._net(xb)
|
| 313 |
+
loss = criterion(preds, yb)
|
| 314 |
+
loss.backward()
|
| 315 |
+
torch.nn.utils.clip_grad_norm_(self._net.parameters(), 1.0)
|
| 316 |
+
optimizer.step()
|
| 317 |
+
epoch_loss += loss.item() * len(xb)
|
| 318 |
+
epoch_loss /= len(train_ds)
|
| 319 |
+
|
| 320 |
+
# Validation
|
| 321 |
+
self._net.eval()
|
| 322 |
+
with torch.no_grad():
|
| 323 |
+
val_preds = self._net(val_X_t)
|
| 324 |
+
val_loss = criterion(val_preds, val_y_t).item()
|
| 325 |
+
self._net.train()
|
| 326 |
+
|
| 327 |
+
scheduler.step(val_loss)
|
| 328 |
+
|
| 329 |
+
if val_loss < best_val_loss - 1e-6:
|
| 330 |
+
best_val_loss = val_loss
|
| 331 |
+
best_state = copy.deepcopy(self._net.state_dict())
|
| 332 |
+
wait = 0
|
| 333 |
+
else:
|
| 334 |
+
wait += 1
|
| 335 |
+
if wait >= self.patience:
|
| 336 |
+
break
|
| 337 |
+
|
| 338 |
+
if best_state is not None:
|
| 339 |
+
self._net.load_state_dict(best_state)
|
| 340 |
+
self._net.eval()
|
| 341 |
+
self._is_fitted = True
|
| 342 |
+
return self
|
| 343 |
+
|
| 344 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 345 |
+
torch, _, _, _, _ = _import_torch()
|
| 346 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 347 |
+
|
| 348 |
+
X = self._prepare(X)
|
| 349 |
+
# If flat, create sequences with padding for early games
|
| 350 |
+
if X.ndim == 2:
|
| 351 |
+
seqs = []
|
| 352 |
+
for i in range(len(X)):
|
| 353 |
+
start = max(0, i - self.seq_len + 1)
|
| 354 |
+
seq = X[start : i + 1]
|
| 355 |
+
if len(seq) < self.seq_len:
|
| 356 |
+
pad = np.zeros((self.seq_len - len(seq), X.shape[1]), dtype=np.float32)
|
| 357 |
+
seq = np.concatenate([pad, seq], axis=0)
|
| 358 |
+
seqs.append(seq)
|
| 359 |
+
X_seq = np.array(seqs, dtype=np.float32)
|
| 360 |
+
else:
|
| 361 |
+
X_seq = X.astype(np.float32)
|
| 362 |
+
|
| 363 |
+
self._net.eval()
|
| 364 |
+
with torch.no_grad():
|
| 365 |
+
preds = self._net(torch.from_numpy(X_seq))
|
| 366 |
+
return preds.numpy()
|
| 367 |
+
|
| 368 |
+
|
| 369 |
+
# ===========================================================================
|
| 370 |
+
# 2. Transformer Attention Model
|
| 371 |
+
# ===========================================================================
|
| 372 |
+
|
| 373 |
+
class TransformerAttentionModel(BaseNBAModel):
|
| 374 |
+
"""
|
| 375 |
+
Self-attention over team performance history.
|
| 376 |
+
|
| 377 |
+
Architecture:
|
| 378 |
+
Linear projection -> Positional encoding ->
|
| 379 |
+
TransformerEncoder (2 layers, 4 heads) ->
|
| 380 |
+
Global average pool -> Dense -> Sigmoid
|
| 381 |
+
|
| 382 |
+
For flat input the model treats each game as one token in a
|
| 383 |
+
sequence of *seq_len* tokens (same sliding-window as LSTM model).
|
| 384 |
+
"""
|
| 385 |
+
|
| 386 |
+
def __init__(
|
| 387 |
+
self,
|
| 388 |
+
seq_len: int = 10,
|
| 389 |
+
d_model: int = 128,
|
| 390 |
+
n_heads: int = 4,
|
| 391 |
+
n_layers: int = 2,
|
| 392 |
+
dim_ff: int = 256,
|
| 393 |
+
dropout: float = 0.2,
|
| 394 |
+
lr: float = 5e-4,
|
| 395 |
+
weight_decay: float = 1e-4,
|
| 396 |
+
batch_size: int = 256,
|
| 397 |
+
epochs: int = 120,
|
| 398 |
+
patience: int = 15,
|
| 399 |
+
**kw,
|
| 400 |
+
):
|
| 401 |
+
super().__init__(
|
| 402 |
+
seq_len=seq_len, d_model=d_model, n_heads=n_heads,
|
| 403 |
+
n_layers=n_layers, dim_ff=dim_ff, dropout=dropout,
|
| 404 |
+
lr=lr, weight_decay=weight_decay, batch_size=batch_size,
|
| 405 |
+
epochs=epochs, patience=patience, **kw,
|
| 406 |
+
)
|
| 407 |
+
self.seq_len = seq_len
|
| 408 |
+
self.d_model = d_model
|
| 409 |
+
self.n_heads = n_heads
|
| 410 |
+
self.n_layers = n_layers
|
| 411 |
+
self.dim_ff = dim_ff
|
| 412 |
+
self.dropout = dropout
|
| 413 |
+
self.lr = lr
|
| 414 |
+
self.weight_decay = weight_decay
|
| 415 |
+
self.batch_size = batch_size
|
| 416 |
+
self.epochs = epochs
|
| 417 |
+
self.patience = patience
|
| 418 |
+
self._net = None
|
| 419 |
+
|
| 420 |
+
@staticmethod
|
| 421 |
+
def _build_net(n_features: int, cfg: dict):
|
| 422 |
+
torch, nn, _, _, _ = _import_torch()
|
| 423 |
+
|
| 424 |
+
class PositionalEncoding(nn.Module):
|
| 425 |
+
"""Sinusoidal positional encoding for game order."""
|
| 426 |
+
def __init__(self, d_model: int, max_len: int = 200):
|
| 427 |
+
super().__init__()
|
| 428 |
+
pe = torch.zeros(max_len, d_model)
|
| 429 |
+
position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
|
| 430 |
+
div_term = torch.exp(
|
| 431 |
+
torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model)
|
| 432 |
+
)
|
| 433 |
+
pe[:, 0::2] = torch.sin(position * div_term)
|
| 434 |
+
pe[:, 1::2] = torch.cos(position * div_term[: d_model // 2]) # handle odd d_model
|
| 435 |
+
pe = pe.unsqueeze(0) # (1, max_len, d_model)
|
| 436 |
+
self.register_buffer("pe", pe)
|
| 437 |
+
|
| 438 |
+
def forward(self, x):
|
| 439 |
+
return x + self.pe[:, : x.size(1), :]
|
| 440 |
+
|
| 441 |
+
class TransformerNet(nn.Module):
|
| 442 |
+
def __init__(self):
|
| 443 |
+
super().__init__()
|
| 444 |
+
self.input_proj = nn.Linear(n_features, cfg["d_model"])
|
| 445 |
+
self.pos_enc = PositionalEncoding(cfg["d_model"], max_len=cfg["seq_len"] + 10)
|
| 446 |
+
self.layer_norm_in = nn.LayerNorm(cfg["d_model"])
|
| 447 |
+
encoder_layer = nn.TransformerEncoderLayer(
|
| 448 |
+
d_model=cfg["d_model"],
|
| 449 |
+
nhead=cfg["n_heads"],
|
| 450 |
+
dim_feedforward=cfg["dim_ff"],
|
| 451 |
+
dropout=cfg["dropout"],
|
| 452 |
+
batch_first=True,
|
| 453 |
+
activation="gelu",
|
| 454 |
+
)
|
| 455 |
+
self.encoder = nn.TransformerEncoder(
|
| 456 |
+
encoder_layer, num_layers=cfg["n_layers"]
|
| 457 |
+
)
|
| 458 |
+
self.dropout = nn.Dropout(cfg["dropout"])
|
| 459 |
+
self.fc1 = nn.Linear(cfg["d_model"], cfg["d_model"] // 2)
|
| 460 |
+
self.gelu = nn.GELU()
|
| 461 |
+
self.fc2 = nn.Linear(cfg["d_model"] // 2, 1)
|
| 462 |
+
|
| 463 |
+
def forward(self, x):
|
| 464 |
+
# x: (batch, seq_len, n_features)
|
| 465 |
+
x = self.input_proj(x)
|
| 466 |
+
x = self.pos_enc(x)
|
| 467 |
+
x = self.layer_norm_in(x)
|
| 468 |
+
x = self.encoder(x)
|
| 469 |
+
# Global average pooling across sequence dim
|
| 470 |
+
x = x.mean(dim=1)
|
| 471 |
+
x = self.dropout(x)
|
| 472 |
+
x = self.gelu(self.fc1(x))
|
| 473 |
+
x = self.dropout(x)
|
| 474 |
+
return torch.sigmoid(self.fc2(x)).squeeze(-1)
|
| 475 |
+
|
| 476 |
+
return TransformerNet()
|
| 477 |
+
|
| 478 |
+
def _make_sequences(self, X: np.ndarray, y: np.ndarray):
|
| 479 |
+
if X.ndim == 3:
|
| 480 |
+
return X, y
|
| 481 |
+
seqs, labels = [], []
|
| 482 |
+
for i in range(self.seq_len - 1, len(X)):
|
| 483 |
+
seqs.append(X[i - self.seq_len + 1 : i + 1])
|
| 484 |
+
labels.append(y[i])
|
| 485 |
+
return np.array(seqs, dtype=np.float32), np.array(labels, dtype=np.float32)
|
| 486 |
+
|
| 487 |
+
def fit(
|
| 488 |
+
self,
|
| 489 |
+
X_train: np.ndarray,
|
| 490 |
+
y_train: np.ndarray,
|
| 491 |
+
X_val: Optional[np.ndarray] = None,
|
| 492 |
+
y_val: Optional[np.ndarray] = None,
|
| 493 |
+
) -> "TransformerAttentionModel":
|
| 494 |
+
torch, nn, optim, DataLoader, TensorDataset = _import_torch()
|
| 495 |
+
|
| 496 |
+
X_train = self._prepare(X_train, fit=True)
|
| 497 |
+
X_train, y_train, X_val, y_val = self._auto_val_split(X_train, y_train, X_val, y_val)
|
| 498 |
+
if X_val is not None:
|
| 499 |
+
X_val = self._prepare(X_val)
|
| 500 |
+
|
| 501 |
+
X_tr_seq, y_tr_seq = self._make_sequences(X_train, y_train)
|
| 502 |
+
X_va_seq, y_va_seq = self._make_sequences(X_val, y_val)
|
| 503 |
+
|
| 504 |
+
n_features = X_tr_seq.shape[2]
|
| 505 |
+
self._net = self._build_net(n_features, {
|
| 506 |
+
"d_model": self.d_model, "n_heads": self.n_heads,
|
| 507 |
+
"n_layers": self.n_layers, "dim_ff": self.dim_ff,
|
| 508 |
+
"dropout": self.dropout, "seq_len": self.seq_len,
|
| 509 |
+
})
|
| 510 |
+
|
| 511 |
+
optimizer = optim.AdamW(
|
| 512 |
+
self._net.parameters(), lr=self.lr, weight_decay=self.weight_decay
|
| 513 |
+
)
|
| 514 |
+
scheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(
|
| 515 |
+
optimizer, T_0=10, T_mult=2, eta_min=1e-6
|
| 516 |
+
)
|
| 517 |
+
criterion = nn.BCELoss()
|
| 518 |
+
|
| 519 |
+
train_ds = TensorDataset(
|
| 520 |
+
torch.from_numpy(X_tr_seq), torch.from_numpy(y_tr_seq)
|
| 521 |
+
)
|
| 522 |
+
train_dl = DataLoader(train_ds, batch_size=self.batch_size, shuffle=True)
|
| 523 |
+
|
| 524 |
+
val_X_t = torch.from_numpy(X_va_seq)
|
| 525 |
+
val_y_t = torch.from_numpy(y_va_seq)
|
| 526 |
+
|
| 527 |
+
best_val_loss = float("inf")
|
| 528 |
+
best_state = None
|
| 529 |
+
wait = 0
|
| 530 |
+
|
| 531 |
+
self._net.train()
|
| 532 |
+
for epoch in range(self.epochs):
|
| 533 |
+
epoch_loss = 0.0
|
| 534 |
+
for xb, yb in train_dl:
|
| 535 |
+
optimizer.zero_grad()
|
| 536 |
+
preds = self._net(xb)
|
| 537 |
+
loss = criterion(preds, yb)
|
| 538 |
+
loss.backward()
|
| 539 |
+
torch.nn.utils.clip_grad_norm_(self._net.parameters(), 1.0)
|
| 540 |
+
optimizer.step()
|
| 541 |
+
epoch_loss += loss.item() * len(xb)
|
| 542 |
+
epoch_loss /= len(train_ds)
|
| 543 |
+
scheduler.step(epoch + epoch_loss) # warm restart input
|
| 544 |
+
|
| 545 |
+
self._net.eval()
|
| 546 |
+
with torch.no_grad():
|
| 547 |
+
val_preds = self._net(val_X_t)
|
| 548 |
+
val_loss = criterion(val_preds, val_y_t).item()
|
| 549 |
+
self._net.train()
|
| 550 |
+
|
| 551 |
+
if val_loss < best_val_loss - 1e-6:
|
| 552 |
+
best_val_loss = val_loss
|
| 553 |
+
best_state = copy.deepcopy(self._net.state_dict())
|
| 554 |
+
wait = 0
|
| 555 |
+
else:
|
| 556 |
+
wait += 1
|
| 557 |
+
if wait >= self.patience:
|
| 558 |
+
break
|
| 559 |
+
|
| 560 |
+
if best_state is not None:
|
| 561 |
+
self._net.load_state_dict(best_state)
|
| 562 |
+
self._net.eval()
|
| 563 |
+
self._is_fitted = True
|
| 564 |
+
return self
|
| 565 |
+
|
| 566 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 567 |
+
torch, _, _, _, _ = _import_torch()
|
| 568 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 569 |
+
|
| 570 |
+
X = self._prepare(X)
|
| 571 |
+
if X.ndim == 2:
|
| 572 |
+
seqs = []
|
| 573 |
+
for i in range(len(X)):
|
| 574 |
+
start = max(0, i - self.seq_len + 1)
|
| 575 |
+
seq = X[start : i + 1]
|
| 576 |
+
if len(seq) < self.seq_len:
|
| 577 |
+
pad = np.zeros((self.seq_len - len(seq), X.shape[1]), dtype=np.float32)
|
| 578 |
+
seq = np.concatenate([pad, seq], axis=0)
|
| 579 |
+
seqs.append(seq)
|
| 580 |
+
X_seq = np.array(seqs, dtype=np.float32)
|
| 581 |
+
else:
|
| 582 |
+
X_seq = X.astype(np.float32)
|
| 583 |
+
|
| 584 |
+
self._net.eval()
|
| 585 |
+
with torch.no_grad():
|
| 586 |
+
preds = self._net(torch.from_numpy(X_seq))
|
| 587 |
+
return preds.numpy()
|
| 588 |
+
|
| 589 |
+
|
| 590 |
+
# ===========================================================================
|
| 591 |
+
# 3. TabNet — Attention-based Tabular Model
|
| 592 |
+
# ===========================================================================
|
| 593 |
+
|
| 594 |
+
class TabNetModel(BaseNBAModel):
|
| 595 |
+
"""
|
| 596 |
+
TabNet (Arik & Pfister 2021) — SOTA attention-based tabular learning.
|
| 597 |
+
|
| 598 |
+
Uses sequential attention to select features at each decision step,
|
| 599 |
+
providing built-in interpretability via attention masks.
|
| 600 |
+
|
| 601 |
+
Wraps pytorch_tabnet.TabNetClassifier with NaN handling and
|
| 602 |
+
early stopping.
|
| 603 |
+
"""
|
| 604 |
+
|
| 605 |
+
def __init__(
|
| 606 |
+
self,
|
| 607 |
+
n_d: int = 32,
|
| 608 |
+
n_a: int = 32,
|
| 609 |
+
n_steps: int = 5,
|
| 610 |
+
gamma: float = 1.5,
|
| 611 |
+
lambda_sparse: float = 1e-4,
|
| 612 |
+
n_independent: int = 2,
|
| 613 |
+
n_shared: int = 2,
|
| 614 |
+
lr: float = 2e-2,
|
| 615 |
+
batch_size: int = 1024,
|
| 616 |
+
virtual_batch_size: int = 256,
|
| 617 |
+
epochs: int = 200,
|
| 618 |
+
patience: int = 20,
|
| 619 |
+
mask_type: str = "entmax",
|
| 620 |
+
**kw,
|
| 621 |
+
):
|
| 622 |
+
super().__init__(
|
| 623 |
+
n_d=n_d, n_a=n_a, n_steps=n_steps, gamma=gamma,
|
| 624 |
+
lambda_sparse=lambda_sparse, n_independent=n_independent,
|
| 625 |
+
n_shared=n_shared, lr=lr, batch_size=batch_size,
|
| 626 |
+
virtual_batch_size=virtual_batch_size, epochs=epochs,
|
| 627 |
+
patience=patience, mask_type=mask_type, **kw,
|
| 628 |
+
)
|
| 629 |
+
self.n_d = n_d
|
| 630 |
+
self.n_a = n_a
|
| 631 |
+
self.n_steps = n_steps
|
| 632 |
+
self.gamma = gamma
|
| 633 |
+
self.lambda_sparse = lambda_sparse
|
| 634 |
+
self.n_independent = n_independent
|
| 635 |
+
self.n_shared = n_shared
|
| 636 |
+
self.lr = lr
|
| 637 |
+
self.batch_size = batch_size
|
| 638 |
+
self.virtual_batch_size = virtual_batch_size
|
| 639 |
+
self.epochs = epochs
|
| 640 |
+
self.patience = patience
|
| 641 |
+
self.mask_type = mask_type
|
| 642 |
+
self._clf = None
|
| 643 |
+
self._feature_importances: Optional[np.ndarray] = None
|
| 644 |
+
|
| 645 |
+
def fit(
|
| 646 |
+
self,
|
| 647 |
+
X_train: np.ndarray,
|
| 648 |
+
y_train: np.ndarray,
|
| 649 |
+
X_val: Optional[np.ndarray] = None,
|
| 650 |
+
y_val: Optional[np.ndarray] = None,
|
| 651 |
+
) -> "TabNetModel":
|
| 652 |
+
from pytorch_tabnet.tab_model import TabNetClassifier
|
| 653 |
+
|
| 654 |
+
X_train = self._impute(X_train, fit=True)
|
| 655 |
+
X_train, y_train, X_val, y_val = self._auto_val_split(X_train, y_train, X_val, y_val)
|
| 656 |
+
if X_val is not None:
|
| 657 |
+
X_val = self._impute(X_val)
|
| 658 |
+
|
| 659 |
+
y_train = y_train.astype(np.int64)
|
| 660 |
+
y_val = y_val.astype(np.int64)
|
| 661 |
+
|
| 662 |
+
self._clf = TabNetClassifier(
|
| 663 |
+
n_d=self.n_d,
|
| 664 |
+
n_a=self.n_a,
|
| 665 |
+
n_steps=self.n_steps,
|
| 666 |
+
gamma=self.gamma,
|
| 667 |
+
lambda_sparse=self.lambda_sparse,
|
| 668 |
+
n_independent=self.n_independent,
|
| 669 |
+
n_shared=self.n_shared,
|
| 670 |
+
optimizer_fn=None, # default Adam
|
| 671 |
+
optimizer_params={"lr": self.lr},
|
| 672 |
+
mask_type=self.mask_type,
|
| 673 |
+
scheduler_fn=None,
|
| 674 |
+
scheduler_params=None,
|
| 675 |
+
verbose=0,
|
| 676 |
+
device_name="cpu",
|
| 677 |
+
)
|
| 678 |
+
|
| 679 |
+
self._clf.fit(
|
| 680 |
+
X_train=X_train,
|
| 681 |
+
y_train=y_train,
|
| 682 |
+
eval_set=[(X_val, y_val)],
|
| 683 |
+
eval_name=["val"],
|
| 684 |
+
eval_metric=["logloss"],
|
| 685 |
+
max_epochs=self.epochs,
|
| 686 |
+
patience=self.patience,
|
| 687 |
+
batch_size=self.batch_size,
|
| 688 |
+
virtual_batch_size=min(self.virtual_batch_size, self.batch_size),
|
| 689 |
+
drop_last=False,
|
| 690 |
+
)
|
| 691 |
+
|
| 692 |
+
self._feature_importances = self._clf.feature_importances_
|
| 693 |
+
self._is_fitted = True
|
| 694 |
+
return self
|
| 695 |
+
|
| 696 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 697 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 698 |
+
X = self._impute(X)
|
| 699 |
+
proba = self._clf.predict_proba(X) # shape (n, 2)
|
| 700 |
+
return proba[:, 1]
|
| 701 |
+
|
| 702 |
+
def get_feature_importances(self) -> Optional[np.ndarray]:
|
| 703 |
+
"""Return TabNet attention-based feature importances."""
|
| 704 |
+
return self._feature_importances
|
| 705 |
+
|
| 706 |
+
def explain(self, X: np.ndarray) -> np.ndarray:
|
| 707 |
+
"""Return per-sample feature attention masks."""
|
| 708 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 709 |
+
X = self._impute(X)
|
| 710 |
+
masks, _ = self._clf.explain(X)
|
| 711 |
+
return masks
|
| 712 |
+
|
| 713 |
+
|
| 714 |
+
# ===========================================================================
|
| 715 |
+
# 4. FT-Transformer (Feature Tokenizer + Transformer)
|
| 716 |
+
# ===========================================================================
|
| 717 |
+
|
| 718 |
+
class FTTransformerModel(BaseNBAModel):
|
| 719 |
+
"""
|
| 720 |
+
FT-Transformer (Gorishniy et al. 2021) — confirmed SOTA for tabular
|
| 721 |
+
data in 2025-2026 benchmarks.
|
| 722 |
+
|
| 723 |
+
Each numerical feature is projected into a *d_token*-dimensional embedding.
|
| 724 |
+
A [CLS] token is prepended. Self-attention across all feature tokens
|
| 725 |
+
captures cross-feature interactions. The [CLS] representation feeds a
|
| 726 |
+
classification head.
|
| 727 |
+
|
| 728 |
+
Because the full 6000+ features would create 6000+ tokens (too large for
|
| 729 |
+
self-attention on CPU), we first apply a learned linear bottleneck to
|
| 730 |
+
reduce to *n_tokens* feature groups.
|
| 731 |
+
"""
|
| 732 |
+
|
| 733 |
+
def __init__(
|
| 734 |
+
self,
|
| 735 |
+
n_tokens: int = 128,
|
| 736 |
+
d_token: int = 64,
|
| 737 |
+
n_heads: int = 4,
|
| 738 |
+
n_layers: int = 3,
|
| 739 |
+
dim_ff: int = 256,
|
| 740 |
+
dropout: float = 0.2,
|
| 741 |
+
attention_dropout: float = 0.1,
|
| 742 |
+
lr: float = 1e-4,
|
| 743 |
+
weight_decay: float = 1e-5,
|
| 744 |
+
batch_size: int = 512,
|
| 745 |
+
epochs: int = 120,
|
| 746 |
+
patience: int = 15,
|
| 747 |
+
**kw,
|
| 748 |
+
):
|
| 749 |
+
super().__init__(
|
| 750 |
+
n_tokens=n_tokens, d_token=d_token, n_heads=n_heads,
|
| 751 |
+
n_layers=n_layers, dim_ff=dim_ff, dropout=dropout,
|
| 752 |
+
attention_dropout=attention_dropout, lr=lr,
|
| 753 |
+
weight_decay=weight_decay, batch_size=batch_size,
|
| 754 |
+
epochs=epochs, patience=patience, **kw,
|
| 755 |
+
)
|
| 756 |
+
self.n_tokens = n_tokens
|
| 757 |
+
self.d_token = d_token
|
| 758 |
+
self.n_heads = n_heads
|
| 759 |
+
self.n_layers = n_layers
|
| 760 |
+
self.dim_ff = dim_ff
|
| 761 |
+
self.dropout = dropout
|
| 762 |
+
self.attention_dropout = attention_dropout
|
| 763 |
+
self.lr = lr
|
| 764 |
+
self.weight_decay = weight_decay
|
| 765 |
+
self.batch_size = batch_size
|
| 766 |
+
self.epochs = epochs
|
| 767 |
+
self.patience = patience
|
| 768 |
+
self._net = None
|
| 769 |
+
|
| 770 |
+
@staticmethod
|
| 771 |
+
def _build_net(n_features: int, cfg: dict):
|
| 772 |
+
torch, nn, _, _, _ = _import_torch()
|
| 773 |
+
|
| 774 |
+
class FTTransformerNet(nn.Module):
|
| 775 |
+
"""
|
| 776 |
+
Feature Tokenizer + Transformer.
|
| 777 |
+
|
| 778 |
+
1) Bottleneck: Linear(n_features -> n_tokens) — group features
|
| 779 |
+
2) Token embed: each of *n_tokens* scalars -> d_token vector
|
| 780 |
+
3) Prepend [CLS] token
|
| 781 |
+
4) TransformerEncoder
|
| 782 |
+
5) [CLS] output -> classification head
|
| 783 |
+
"""
|
| 784 |
+
|
| 785 |
+
def __init__(self):
|
| 786 |
+
super().__init__()
|
| 787 |
+
n_tok = cfg["n_tokens"]
|
| 788 |
+
d_tok = cfg["d_token"]
|
| 789 |
+
|
| 790 |
+
# Bottleneck projection: reduce 6000 features to n_tokens groups
|
| 791 |
+
self.bottleneck = nn.Linear(n_features, n_tok)
|
| 792 |
+
self.bn_norm = nn.LayerNorm(n_tok)
|
| 793 |
+
|
| 794 |
+
# Per-token embedding: each scalar -> d_token vector
|
| 795 |
+
# Implemented as a shared Linear(1 -> d_token) + per-token bias
|
| 796 |
+
self.token_weight = nn.Parameter(torch.randn(n_tok, d_tok) * 0.02)
|
| 797 |
+
self.token_bias = nn.Parameter(torch.zeros(n_tok, d_tok))
|
| 798 |
+
|
| 799 |
+
# [CLS] token
|
| 800 |
+
self.cls_token = nn.Parameter(torch.randn(1, 1, d_tok) * 0.02)
|
| 801 |
+
|
| 802 |
+
# Transformer
|
| 803 |
+
self.layer_norm = nn.LayerNorm(d_tok)
|
| 804 |
+
encoder_layer = nn.TransformerEncoderLayer(
|
| 805 |
+
d_model=d_tok,
|
| 806 |
+
nhead=cfg["n_heads"],
|
| 807 |
+
dim_feedforward=cfg["dim_ff"],
|
| 808 |
+
dropout=cfg["dropout"],
|
| 809 |
+
batch_first=True,
|
| 810 |
+
activation="gelu",
|
| 811 |
+
)
|
| 812 |
+
self.encoder = nn.TransformerEncoder(
|
| 813 |
+
encoder_layer, num_layers=cfg["n_layers"]
|
| 814 |
+
)
|
| 815 |
+
|
| 816 |
+
# Head
|
| 817 |
+
self.head = nn.Sequential(
|
| 818 |
+
nn.LayerNorm(d_tok),
|
| 819 |
+
nn.Linear(d_tok, d_tok // 2),
|
| 820 |
+
nn.GELU(),
|
| 821 |
+
nn.Dropout(cfg["dropout"]),
|
| 822 |
+
nn.Linear(d_tok // 2, 1),
|
| 823 |
+
)
|
| 824 |
+
|
| 825 |
+
def forward(self, x):
|
| 826 |
+
# x: (batch, n_features)
|
| 827 |
+
batch_size = x.size(0)
|
| 828 |
+
|
| 829 |
+
# Bottleneck: (batch, n_features) -> (batch, n_tokens)
|
| 830 |
+
x = self.bn_norm(self.bottleneck(x))
|
| 831 |
+
|
| 832 |
+
# Token embedding: (batch, n_tokens) -> (batch, n_tokens, d_token)
|
| 833 |
+
# x_i * weight_i + bias_i for each token
|
| 834 |
+
x = x.unsqueeze(-1) * self.token_weight.unsqueeze(0) + self.token_bias.unsqueeze(0)
|
| 835 |
+
|
| 836 |
+
# Prepend [CLS]
|
| 837 |
+
cls = self.cls_token.expand(batch_size, -1, -1)
|
| 838 |
+
x = torch.cat([cls, x], dim=1) # (batch, 1 + n_tokens, d_token)
|
| 839 |
+
|
| 840 |
+
x = self.layer_norm(x)
|
| 841 |
+
x = self.encoder(x)
|
| 842 |
+
|
| 843 |
+
# Extract [CLS] output
|
| 844 |
+
cls_out = x[:, 0, :]
|
| 845 |
+
return torch.sigmoid(self.head(cls_out)).squeeze(-1)
|
| 846 |
+
|
| 847 |
+
return FTTransformerNet()
|
| 848 |
+
|
| 849 |
+
def fit(
|
| 850 |
+
self,
|
| 851 |
+
X_train: np.ndarray,
|
| 852 |
+
y_train: np.ndarray,
|
| 853 |
+
X_val: Optional[np.ndarray] = None,
|
| 854 |
+
y_val: Optional[np.ndarray] = None,
|
| 855 |
+
) -> "FTTransformerModel":
|
| 856 |
+
torch, nn, optim, DataLoader, TensorDataset = _import_torch()
|
| 857 |
+
|
| 858 |
+
X_train = self._prepare(X_train, fit=True)
|
| 859 |
+
X_train, y_train, X_val, y_val = self._auto_val_split(X_train, y_train, X_val, y_val)
|
| 860 |
+
if X_val is not None:
|
| 861 |
+
X_val = self._prepare(X_val)
|
| 862 |
+
|
| 863 |
+
y_train = y_train.astype(np.float32)
|
| 864 |
+
y_val = y_val.astype(np.float32)
|
| 865 |
+
|
| 866 |
+
n_features = X_train.shape[1]
|
| 867 |
+
self._net = self._build_net(n_features, {
|
| 868 |
+
"n_tokens": min(self.n_tokens, n_features),
|
| 869 |
+
"d_token": self.d_token,
|
| 870 |
+
"n_heads": self.n_heads,
|
| 871 |
+
"n_layers": self.n_layers,
|
| 872 |
+
"dim_ff": self.dim_ff,
|
| 873 |
+
"dropout": self.dropout,
|
| 874 |
+
})
|
| 875 |
+
|
| 876 |
+
optimizer = optim.AdamW(
|
| 877 |
+
self._net.parameters(), lr=self.lr, weight_decay=self.weight_decay
|
| 878 |
+
)
|
| 879 |
+
scheduler = optim.lr_scheduler.OneCycleLR(
|
| 880 |
+
optimizer, max_lr=self.lr * 10, total_steps=self.epochs,
|
| 881 |
+
pct_start=0.1, anneal_strategy="cos",
|
| 882 |
+
)
|
| 883 |
+
criterion = nn.BCELoss()
|
| 884 |
+
|
| 885 |
+
train_ds = TensorDataset(
|
| 886 |
+
torch.from_numpy(X_train), torch.from_numpy(y_train)
|
| 887 |
+
)
|
| 888 |
+
train_dl = DataLoader(train_ds, batch_size=self.batch_size, shuffle=True)
|
| 889 |
+
|
| 890 |
+
val_X_t = torch.from_numpy(X_val)
|
| 891 |
+
val_y_t = torch.from_numpy(y_val)
|
| 892 |
+
|
| 893 |
+
best_val_loss = float("inf")
|
| 894 |
+
best_state = None
|
| 895 |
+
wait = 0
|
| 896 |
+
|
| 897 |
+
self._net.train()
|
| 898 |
+
for epoch in range(self.epochs):
|
| 899 |
+
epoch_loss = 0.0
|
| 900 |
+
for xb, yb in train_dl:
|
| 901 |
+
optimizer.zero_grad()
|
| 902 |
+
preds = self._net(xb)
|
| 903 |
+
loss = criterion(preds, yb)
|
| 904 |
+
loss.backward()
|
| 905 |
+
torch.nn.utils.clip_grad_norm_(self._net.parameters(), 1.0)
|
| 906 |
+
optimizer.step()
|
| 907 |
+
epoch_loss += loss.item() * len(xb)
|
| 908 |
+
epoch_loss /= len(train_ds)
|
| 909 |
+
scheduler.step()
|
| 910 |
+
|
| 911 |
+
self._net.eval()
|
| 912 |
+
with torch.no_grad():
|
| 913 |
+
val_preds = self._net(val_X_t)
|
| 914 |
+
val_loss = criterion(val_preds, val_y_t).item()
|
| 915 |
+
self._net.train()
|
| 916 |
+
|
| 917 |
+
if val_loss < best_val_loss - 1e-6:
|
| 918 |
+
best_val_loss = val_loss
|
| 919 |
+
best_state = copy.deepcopy(self._net.state_dict())
|
| 920 |
+
wait = 0
|
| 921 |
+
else:
|
| 922 |
+
wait += 1
|
| 923 |
+
if wait >= self.patience:
|
| 924 |
+
break
|
| 925 |
+
|
| 926 |
+
if best_state is not None:
|
| 927 |
+
self._net.load_state_dict(best_state)
|
| 928 |
+
self._net.eval()
|
| 929 |
+
self._is_fitted = True
|
| 930 |
+
return self
|
| 931 |
+
|
| 932 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 933 |
+
torch, _, _, _, _ = _import_torch()
|
| 934 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 935 |
+
|
| 936 |
+
X = self._prepare(X)
|
| 937 |
+
X_t = torch.from_numpy(X)
|
| 938 |
+
|
| 939 |
+
self._net.eval()
|
| 940 |
+
# Batch to avoid OOM on large inputs
|
| 941 |
+
preds_list = []
|
| 942 |
+
bs = self.batch_size
|
| 943 |
+
for i in range(0, len(X_t), bs):
|
| 944 |
+
with torch.no_grad():
|
| 945 |
+
p = self._net(X_t[i : i + bs])
|
| 946 |
+
preds_list.append(p.numpy())
|
| 947 |
+
return np.concatenate(preds_list)
|
| 948 |
+
|
| 949 |
+
|
| 950 |
+
# ===========================================================================
|
| 951 |
+
# 5. Deep Ensemble
|
| 952 |
+
# ===========================================================================
|
| 953 |
+
|
| 954 |
+
class DeepEnsemble(BaseNBAModel):
|
| 955 |
+
"""
|
| 956 |
+
Train N independent neural networks with different random seeds.
|
| 957 |
+
|
| 958 |
+
Average their predictions for:
|
| 959 |
+
- Better calibration (ensemble smoothing)
|
| 960 |
+
- Uncertainty estimation (prediction variance)
|
| 961 |
+
|
| 962 |
+
Each member is a simple but effective MLP with skip connections (ResNet-style),
|
| 963 |
+
which is the 2025 consensus best architecture for tabular deep learning
|
| 964 |
+
when ensembled (Kadra et al. 2021 "Well-Tuned Simple Nets").
|
| 965 |
+
"""
|
| 966 |
+
|
| 967 |
+
def __init__(
|
| 968 |
+
self,
|
| 969 |
+
n_members: int = 10,
|
| 970 |
+
hidden_dims: Tuple[int, ...] = (512, 256, 128),
|
| 971 |
+
dropout: float = 0.3,
|
| 972 |
+
lr: float = 1e-3,
|
| 973 |
+
weight_decay: float = 1e-4,
|
| 974 |
+
batch_size: int = 512,
|
| 975 |
+
epochs: int = 100,
|
| 976 |
+
patience: int = 12,
|
| 977 |
+
**kw,
|
| 978 |
+
):
|
| 979 |
+
super().__init__(
|
| 980 |
+
n_members=n_members, hidden_dims=list(hidden_dims),
|
| 981 |
+
dropout=dropout, lr=lr, weight_decay=weight_decay,
|
| 982 |
+
batch_size=batch_size, epochs=epochs, patience=patience, **kw,
|
| 983 |
+
)
|
| 984 |
+
self.n_members = n_members
|
| 985 |
+
self.hidden_dims = hidden_dims
|
| 986 |
+
self.dropout = dropout
|
| 987 |
+
self.lr = lr
|
| 988 |
+
self.weight_decay = weight_decay
|
| 989 |
+
self.batch_size = batch_size
|
| 990 |
+
self.epochs = epochs
|
| 991 |
+
self.patience = patience
|
| 992 |
+
self._members: List = []
|
| 993 |
+
|
| 994 |
+
@staticmethod
|
| 995 |
+
def _build_mlp(n_features: int, hidden_dims: Tuple[int, ...], dropout: float, seed: int):
|
| 996 |
+
"""Build one ResNet-style MLP member."""
|
| 997 |
+
torch, nn, _, _, _ = _import_torch()
|
| 998 |
+
torch.manual_seed(seed)
|
| 999 |
+
|
| 1000 |
+
class ResBlock(nn.Module):
|
| 1001 |
+
"""Pre-activation residual block."""
|
| 1002 |
+
def __init__(self, dim: int, drop: float):
|
| 1003 |
+
super().__init__()
|
| 1004 |
+
self.net = nn.Sequential(
|
| 1005 |
+
nn.LayerNorm(dim),
|
| 1006 |
+
nn.GELU(),
|
| 1007 |
+
nn.Linear(dim, dim),
|
| 1008 |
+
nn.Dropout(drop),
|
| 1009 |
+
nn.LayerNorm(dim),
|
| 1010 |
+
nn.GELU(),
|
| 1011 |
+
nn.Linear(dim, dim),
|
| 1012 |
+
nn.Dropout(drop),
|
| 1013 |
+
)
|
| 1014 |
+
|
| 1015 |
+
def forward(self, x):
|
| 1016 |
+
return x + self.net(x)
|
| 1017 |
+
|
| 1018 |
+
layers = []
|
| 1019 |
+
in_dim = n_features
|
| 1020 |
+
for h_dim in hidden_dims:
|
| 1021 |
+
layers.append(nn.Linear(in_dim, h_dim))
|
| 1022 |
+
layers.append(nn.GELU())
|
| 1023 |
+
layers.append(nn.Dropout(dropout))
|
| 1024 |
+
# Add residual block at each hidden layer
|
| 1025 |
+
layers.append(ResBlock(h_dim, dropout))
|
| 1026 |
+
in_dim = h_dim
|
| 1027 |
+
layers.append(nn.Linear(in_dim, 1))
|
| 1028 |
+
|
| 1029 |
+
class EnsembleMLP(nn.Module):
|
| 1030 |
+
def __init__(self, layer_list):
|
| 1031 |
+
super().__init__()
|
| 1032 |
+
self.net = nn.Sequential(*layer_list)
|
| 1033 |
+
|
| 1034 |
+
def forward(self, x):
|
| 1035 |
+
return torch.sigmoid(self.net(x)).squeeze(-1)
|
| 1036 |
+
|
| 1037 |
+
return EnsembleMLP(layers)
|
| 1038 |
+
|
| 1039 |
+
def fit(
|
| 1040 |
+
self,
|
| 1041 |
+
X_train: np.ndarray,
|
| 1042 |
+
y_train: np.ndarray,
|
| 1043 |
+
X_val: Optional[np.ndarray] = None,
|
| 1044 |
+
y_val: Optional[np.ndarray] = None,
|
| 1045 |
+
) -> "DeepEnsemble":
|
| 1046 |
+
torch, nn, optim, DataLoader, TensorDataset = _import_torch()
|
| 1047 |
+
|
| 1048 |
+
X_train = self._prepare(X_train, fit=True)
|
| 1049 |
+
X_train, y_train, X_val, y_val = self._auto_val_split(X_train, y_train, X_val, y_val)
|
| 1050 |
+
if X_val is not None:
|
| 1051 |
+
X_val = self._prepare(X_val)
|
| 1052 |
+
|
| 1053 |
+
y_train = y_train.astype(np.float32)
|
| 1054 |
+
y_val = y_val.astype(np.float32)
|
| 1055 |
+
n_features = X_train.shape[1]
|
| 1056 |
+
|
| 1057 |
+
val_X_t = torch.from_numpy(X_val)
|
| 1058 |
+
val_y_t = torch.from_numpy(y_val)
|
| 1059 |
+
criterion = nn.BCELoss()
|
| 1060 |
+
|
| 1061 |
+
self._members = []
|
| 1062 |
+
for member_idx in range(self.n_members):
|
| 1063 |
+
seed = 42 + member_idx * 1337
|
| 1064 |
+
net = self._build_mlp(n_features, self.hidden_dims, self.dropout, seed)
|
| 1065 |
+
|
| 1066 |
+
# Each member gets a different random seed for data shuffling too
|
| 1067 |
+
torch.manual_seed(seed)
|
| 1068 |
+
np.random.seed(seed)
|
| 1069 |
+
|
| 1070 |
+
optimizer = optim.AdamW(
|
| 1071 |
+
net.parameters(), lr=self.lr, weight_decay=self.weight_decay
|
| 1072 |
+
)
|
| 1073 |
+
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
|
| 1074 |
+
optimizer, mode="min", factor=0.5, patience=5, min_lr=1e-6
|
| 1075 |
+
)
|
| 1076 |
+
|
| 1077 |
+
train_ds = TensorDataset(
|
| 1078 |
+
torch.from_numpy(X_train), torch.from_numpy(y_train)
|
| 1079 |
+
)
|
| 1080 |
+
train_dl = DataLoader(train_ds, batch_size=self.batch_size, shuffle=True)
|
| 1081 |
+
|
| 1082 |
+
best_val_loss = float("inf")
|
| 1083 |
+
best_state = None
|
| 1084 |
+
wait = 0
|
| 1085 |
+
|
| 1086 |
+
net.train()
|
| 1087 |
+
for epoch in range(self.epochs):
|
| 1088 |
+
for xb, yb in train_dl:
|
| 1089 |
+
optimizer.zero_grad()
|
| 1090 |
+
preds = net(xb)
|
| 1091 |
+
loss = criterion(preds, yb)
|
| 1092 |
+
loss.backward()
|
| 1093 |
+
torch.nn.utils.clip_grad_norm_(net.parameters(), 1.0)
|
| 1094 |
+
optimizer.step()
|
| 1095 |
+
|
| 1096 |
+
net.eval()
|
| 1097 |
+
with torch.no_grad():
|
| 1098 |
+
vp = net(val_X_t)
|
| 1099 |
+
vl = criterion(vp, val_y_t).item()
|
| 1100 |
+
net.train()
|
| 1101 |
+
scheduler.step(vl)
|
| 1102 |
+
|
| 1103 |
+
if vl < best_val_loss - 1e-6:
|
| 1104 |
+
best_val_loss = vl
|
| 1105 |
+
best_state = copy.deepcopy(net.state_dict())
|
| 1106 |
+
wait = 0
|
| 1107 |
+
else:
|
| 1108 |
+
wait += 1
|
| 1109 |
+
if wait >= self.patience:
|
| 1110 |
+
break
|
| 1111 |
+
|
| 1112 |
+
if best_state is not None:
|
| 1113 |
+
net.load_state_dict(best_state)
|
| 1114 |
+
net.eval()
|
| 1115 |
+
self._members.append(net)
|
| 1116 |
+
|
| 1117 |
+
self._is_fitted = True
|
| 1118 |
+
return self
|
| 1119 |
+
|
| 1120 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 1121 |
+
"""Return mean prediction across ensemble members."""
|
| 1122 |
+
torch, _, _, _, _ = _import_torch()
|
| 1123 |
+
assert self._is_fitted and self._members, "Model not fitted yet"
|
| 1124 |
+
|
| 1125 |
+
X = self._prepare(X)
|
| 1126 |
+
X_t = torch.from_numpy(X)
|
| 1127 |
+
|
| 1128 |
+
all_preds = []
|
| 1129 |
+
for net in self._members:
|
| 1130 |
+
net.eval()
|
| 1131 |
+
with torch.no_grad():
|
| 1132 |
+
p = net(X_t).numpy()
|
| 1133 |
+
all_preds.append(p)
|
| 1134 |
+
|
| 1135 |
+
return np.mean(all_preds, axis=0)
|
| 1136 |
+
|
| 1137 |
+
def predict_uncertainty(self, X: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
|
| 1138 |
+
"""
|
| 1139 |
+
Return (mean_prediction, std_prediction) across ensemble members.
|
| 1140 |
+
High std = high model uncertainty = less confident prediction.
|
| 1141 |
+
"""
|
| 1142 |
+
torch, _, _, _, _ = _import_torch()
|
| 1143 |
+
assert self._is_fitted and self._members, "Model not fitted yet"
|
| 1144 |
+
|
| 1145 |
+
X = self._prepare(X)
|
| 1146 |
+
X_t = torch.from_numpy(X)
|
| 1147 |
+
|
| 1148 |
+
all_preds = []
|
| 1149 |
+
for net in self._members:
|
| 1150 |
+
net.eval()
|
| 1151 |
+
with torch.no_grad():
|
| 1152 |
+
p = net(X_t).numpy()
|
| 1153 |
+
all_preds.append(p)
|
| 1154 |
+
|
| 1155 |
+
stacked = np.array(all_preds) # (n_members, n_samples)
|
| 1156 |
+
return stacked.mean(axis=0), stacked.std(axis=0)
|
| 1157 |
+
|
| 1158 |
+
|
| 1159 |
+
# ===========================================================================
|
| 1160 |
+
# 6. Conformal Prediction Wrapper
|
| 1161 |
+
# ===========================================================================
|
| 1162 |
+
|
| 1163 |
+
class ConformalPredictionWrapper(BaseNBAModel):
|
| 1164 |
+
"""
|
| 1165 |
+
Wraps ANY model to provide calibrated prediction intervals with
|
| 1166 |
+
guaranteed coverage.
|
| 1167 |
+
|
| 1168 |
+
Uses split conformal prediction:
|
| 1169 |
+
1. Train base model on training set
|
| 1170 |
+
2. Compute non-conformity scores on calibration holdout
|
| 1171 |
+
3. At inference, use quantile of scores to produce prediction sets
|
| 1172 |
+
|
| 1173 |
+
For binary classification:
|
| 1174 |
+
- Returns P(home_win) from base model (point prediction)
|
| 1175 |
+
- Also provides prediction_set() that returns {0}, {1}, or {0,1}
|
| 1176 |
+
with guaranteed marginal coverage >= (1 - alpha)
|
| 1177 |
+
"""
|
| 1178 |
+
|
| 1179 |
+
def __init__(
|
| 1180 |
+
self,
|
| 1181 |
+
base_model: BaseNBAModel,
|
| 1182 |
+
alpha: float = 0.10,
|
| 1183 |
+
cal_fraction: float = 0.20,
|
| 1184 |
+
**kw,
|
| 1185 |
+
):
|
| 1186 |
+
super().__init__(alpha=alpha, cal_fraction=cal_fraction, **kw)
|
| 1187 |
+
self.base_model = base_model
|
| 1188 |
+
self.alpha = alpha
|
| 1189 |
+
self.cal_fraction = cal_fraction
|
| 1190 |
+
self._qhat: Optional[float] = None
|
| 1191 |
+
self._cal_scores: Optional[np.ndarray] = None
|
| 1192 |
+
|
| 1193 |
+
def fit(
|
| 1194 |
+
self,
|
| 1195 |
+
X_train: np.ndarray,
|
| 1196 |
+
y_train: np.ndarray,
|
| 1197 |
+
X_val: Optional[np.ndarray] = None,
|
| 1198 |
+
y_val: Optional[np.ndarray] = None,
|
| 1199 |
+
) -> "ConformalPredictionWrapper":
|
| 1200 |
+
"""
|
| 1201 |
+
Split data into proper-training and calibration sets.
|
| 1202 |
+
Train base model on proper-training, compute conformal scores on calibration.
|
| 1203 |
+
"""
|
| 1204 |
+
n = len(X_train)
|
| 1205 |
+
cal_size = int(n * self.cal_fraction)
|
| 1206 |
+
# Use the LAST cal_size samples for calibration (time-ordered)
|
| 1207 |
+
X_proper = X_train[: n - cal_size]
|
| 1208 |
+
y_proper = y_train[: n - cal_size]
|
| 1209 |
+
X_cal = X_train[n - cal_size :]
|
| 1210 |
+
y_cal = y_train[n - cal_size :]
|
| 1211 |
+
|
| 1212 |
+
# Train base model
|
| 1213 |
+
self.base_model.fit(X_proper, y_proper, X_val, y_val)
|
| 1214 |
+
|
| 1215 |
+
# Compute non-conformity scores on calibration set
|
| 1216 |
+
cal_probs = self.base_model.predict_proba(X_cal)
|
| 1217 |
+
# Score = 1 - P(true_class)
|
| 1218 |
+
scores = np.where(y_cal == 1, 1.0 - cal_probs, cal_probs)
|
| 1219 |
+
self._cal_scores = np.sort(scores)
|
| 1220 |
+
|
| 1221 |
+
# Quantile for desired coverage
|
| 1222 |
+
n_cal = len(self._cal_scores)
|
| 1223 |
+
level = np.ceil((1.0 - self.alpha) * (n_cal + 1)) / n_cal
|
| 1224 |
+
level = min(level, 1.0)
|
| 1225 |
+
self._qhat = np.quantile(self._cal_scores, level, method="higher")
|
| 1226 |
+
|
| 1227 |
+
self._is_fitted = True
|
| 1228 |
+
return self
|
| 1229 |
+
|
| 1230 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 1231 |
+
"""Return point predictions from base model."""
|
| 1232 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 1233 |
+
return self.base_model.predict_proba(X)
|
| 1234 |
+
|
| 1235 |
+
def predict_sets(self, X: np.ndarray) -> List[set]:
|
| 1236 |
+
"""
|
| 1237 |
+
Return prediction sets with guaranteed (1-alpha) coverage.
|
| 1238 |
+
|
| 1239 |
+
Each set is one of:
|
| 1240 |
+
- {1} — confident home win
|
| 1241 |
+
- {0} — confident away win
|
| 1242 |
+
- {0, 1} — uncertain (both plausible)
|
| 1243 |
+
"""
|
| 1244 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 1245 |
+
probs = self.base_model.predict_proba(X)
|
| 1246 |
+
sets = []
|
| 1247 |
+
for p in probs:
|
| 1248 |
+
s = set()
|
| 1249 |
+
# Include class 1 if score would be <= qhat
|
| 1250 |
+
if 1.0 - p <= self._qhat:
|
| 1251 |
+
s.add(1)
|
| 1252 |
+
# Include class 0 if score would be <= qhat
|
| 1253 |
+
if p <= self._qhat:
|
| 1254 |
+
s.add(0)
|
| 1255 |
+
if not s:
|
| 1256 |
+
# Shouldn't happen, but include most likely
|
| 1257 |
+
s.add(1 if p >= 0.5 else 0)
|
| 1258 |
+
sets.append(s)
|
| 1259 |
+
return sets
|
| 1260 |
+
|
| 1261 |
+
def predict_intervals(self, X: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
|
| 1262 |
+
"""
|
| 1263 |
+
Return (lower_bound, upper_bound) calibrated probability intervals.
|
| 1264 |
+
|
| 1265 |
+
Width of interval reflects model uncertainty after conformal calibration.
|
| 1266 |
+
"""
|
| 1267 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 1268 |
+
probs = self.base_model.predict_proba(X)
|
| 1269 |
+
lower = np.clip(probs - self._qhat, 0.0, 1.0)
|
| 1270 |
+
upper = np.clip(probs + self._qhat, 0.0, 1.0)
|
| 1271 |
+
return lower, upper
|
| 1272 |
+
|
| 1273 |
+
def get_params(self) -> Dict[str, Any]:
|
| 1274 |
+
base_params = self.base_model.get_params()
|
| 1275 |
+
return {
|
| 1276 |
+
"wrapper": "conformal",
|
| 1277 |
+
"alpha": self.alpha,
|
| 1278 |
+
"cal_fraction": self.cal_fraction,
|
| 1279 |
+
"qhat": float(self._qhat) if self._qhat is not None else None,
|
| 1280 |
+
"base_model": base_params,
|
| 1281 |
+
}
|
| 1282 |
+
|
| 1283 |
+
|
| 1284 |
+
# ===========================================================================
|
| 1285 |
+
# 7. AutoGluon Ensemble
|
| 1286 |
+
# ===========================================================================
|
| 1287 |
+
|
| 1288 |
+
class AutoGluonEnsemble(BaseNBAModel):
|
| 1289 |
+
"""
|
| 1290 |
+
AutoGluon Tabular — auto-search and stack hundreds of model configurations.
|
| 1291 |
+
|
| 1292 |
+
Time-budgeted: runs for *max_time* seconds, tries GBMs, neural nets,
|
| 1293 |
+
linear models, k-NN, then stacks the best ones.
|
| 1294 |
+
|
| 1295 |
+
Presets: "best_quality" = maximum stacking/bagging (slow but best),
|
| 1296 |
+
"good_quality" = reasonable speed/quality trade-off,
|
| 1297 |
+
"medium_quality" = fastest.
|
| 1298 |
+
"""
|
| 1299 |
+
|
| 1300 |
+
def __init__(
|
| 1301 |
+
self,
|
| 1302 |
+
max_time: int = 3600,
|
| 1303 |
+
preset: str = "best_quality",
|
| 1304 |
+
eval_metric: str = "log_loss",
|
| 1305 |
+
num_bag_folds: int = 5,
|
| 1306 |
+
num_stack_levels: int = 1,
|
| 1307 |
+
verbosity: int = 1,
|
| 1308 |
+
**kw,
|
| 1309 |
+
):
|
| 1310 |
+
super().__init__(
|
| 1311 |
+
max_time=max_time, preset=preset, eval_metric=eval_metric,
|
| 1312 |
+
num_bag_folds=num_bag_folds, num_stack_levels=num_stack_levels,
|
| 1313 |
+
verbosity=verbosity, **kw,
|
| 1314 |
+
)
|
| 1315 |
+
self.max_time = max_time
|
| 1316 |
+
self.preset = preset
|
| 1317 |
+
self.eval_metric = eval_metric
|
| 1318 |
+
self.num_bag_folds = num_bag_folds
|
| 1319 |
+
self.num_stack_levels = num_stack_levels
|
| 1320 |
+
self.verbosity = verbosity
|
| 1321 |
+
self._predictor = None
|
| 1322 |
+
|
| 1323 |
+
def fit(
|
| 1324 |
+
self,
|
| 1325 |
+
X_train: np.ndarray,
|
| 1326 |
+
y_train: np.ndarray,
|
| 1327 |
+
X_val: Optional[np.ndarray] = None,
|
| 1328 |
+
y_val: Optional[np.ndarray] = None,
|
| 1329 |
+
) -> "AutoGluonEnsemble":
|
| 1330 |
+
try:
|
| 1331 |
+
from autogluon.tabular import TabularPredictor
|
| 1332 |
+
import pandas as pd
|
| 1333 |
+
except ImportError:
|
| 1334 |
+
raise ImportError(
|
| 1335 |
+
"autogluon.tabular not installed. Install with: "
|
| 1336 |
+
"pip install autogluon.tabular"
|
| 1337 |
+
)
|
| 1338 |
+
|
| 1339 |
+
X_train = self._impute(X_train, fit=True)
|
| 1340 |
+
|
| 1341 |
+
# Build DataFrame with feature columns + label
|
| 1342 |
+
n_features = X_train.shape[1]
|
| 1343 |
+
col_names = [f"f_{i}" for i in range(n_features)]
|
| 1344 |
+
df_train = pd.DataFrame(X_train, columns=col_names)
|
| 1345 |
+
df_train["label"] = y_train.astype(int)
|
| 1346 |
+
|
| 1347 |
+
# Validation data (optional tuning set)
|
| 1348 |
+
df_val = None
|
| 1349 |
+
if X_val is not None and y_val is not None:
|
| 1350 |
+
X_val = self._impute(X_val)
|
| 1351 |
+
df_val = pd.DataFrame(X_val, columns=col_names)
|
| 1352 |
+
df_val["label"] = y_val.astype(int)
|
| 1353 |
+
|
| 1354 |
+
self._col_names = col_names
|
| 1355 |
+
|
| 1356 |
+
self._predictor = TabularPredictor(
|
| 1357 |
+
label="label",
|
| 1358 |
+
eval_metric=self.eval_metric,
|
| 1359 |
+
problem_type="binary",
|
| 1360 |
+
verbosity=self.verbosity,
|
| 1361 |
+
)
|
| 1362 |
+
|
| 1363 |
+
fit_kwargs = {
|
| 1364 |
+
"train_data": df_train,
|
| 1365 |
+
"time_limit": self.max_time,
|
| 1366 |
+
"presets": self.preset,
|
| 1367 |
+
"num_bag_folds": self.num_bag_folds,
|
| 1368 |
+
"num_stack_levels": self.num_stack_levels,
|
| 1369 |
+
}
|
| 1370 |
+
if df_val is not None:
|
| 1371 |
+
fit_kwargs["tuning_data"] = df_val
|
| 1372 |
+
|
| 1373 |
+
self._predictor.fit(**fit_kwargs)
|
| 1374 |
+
self._is_fitted = True
|
| 1375 |
+
return self
|
| 1376 |
+
|
| 1377 |
+
def predict_proba(self, X: np.ndarray) -> np.ndarray:
|
| 1378 |
+
import pandas as pd
|
| 1379 |
+
|
| 1380 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 1381 |
+
X = self._impute(X)
|
| 1382 |
+
df = pd.DataFrame(X, columns=self._col_names)
|
| 1383 |
+
proba = self._predictor.predict_proba(df)
|
| 1384 |
+
# Returns DataFrame with columns 0, 1 — we want P(class=1)
|
| 1385 |
+
if isinstance(proba, pd.DataFrame):
|
| 1386 |
+
return proba[1].values
|
| 1387 |
+
return proba
|
| 1388 |
+
|
| 1389 |
+
def leaderboard(self):
|
| 1390 |
+
"""Return AutoGluon model leaderboard."""
|
| 1391 |
+
assert self._is_fitted, "Model not fitted yet"
|
| 1392 |
+
return self._predictor.leaderboard(silent=True)
|
| 1393 |
+
|
| 1394 |
+
def feature_importance(self, X: np.ndarray, y: np.ndarray) -> "pd.DataFrame":
|
| 1395 |
+
"""Return permutation feature importance."""
|
| 1396 |
+
import pandas as pd
|
| 1397 |
+
|
| 1398 |
+
X = self._impute(X)
|
| 1399 |
+
df = pd.DataFrame(X, columns=self._col_names)
|
| 1400 |
+
df["label"] = y.astype(int)
|
| 1401 |
+
return self._predictor.feature_importance(df)
|
| 1402 |
+
|
| 1403 |
+
def save(self, path: Union[str, Path]) -> None:
|
| 1404 |
+
"""AutoGluon has its own save mechanism."""
|
| 1405 |
+
path = Path(path)
|
| 1406 |
+
path.mkdir(parents=True, exist_ok=True)
|
| 1407 |
+
if self._predictor is not None:
|
| 1408 |
+
self._predictor.save(str(path / "autogluon_predictor"))
|
| 1409 |
+
# Save wrapper state
|
| 1410 |
+
state = {
|
| 1411 |
+
"params": self.params,
|
| 1412 |
+
"_col_names": getattr(self, "_col_names", None),
|
| 1413 |
+
"_feature_medians": self._feature_medians.tolist() if self._feature_medians is not None else None,
|
| 1414 |
+
"_is_fitted": self._is_fitted,
|
| 1415 |
+
}
|
| 1416 |
+
with open(path / "wrapper_state.json", "w") as f:
|
| 1417 |
+
json.dump(state, f)
|
| 1418 |
+
|
| 1419 |
+
@classmethod
|
| 1420 |
+
def load(cls, path: Union[str, Path]) -> "AutoGluonEnsemble":
|
| 1421 |
+
from autogluon.tabular import TabularPredictor
|
| 1422 |
+
|
| 1423 |
+
path = Path(path)
|
| 1424 |
+
with open(path / "wrapper_state.json") as f:
|
| 1425 |
+
state = json.load(f)
|
| 1426 |
+
|
| 1427 |
+
obj = cls(**state["params"])
|
| 1428 |
+
obj._col_names = state["_col_names"]
|
| 1429 |
+
if state["_feature_medians"] is not None:
|
| 1430 |
+
obj._feature_medians = np.array(state["_feature_medians"], dtype=np.float32)
|
| 1431 |
+
obj._predictor = TabularPredictor.load(str(path / "autogluon_predictor"))
|
| 1432 |
+
obj._is_fitted = state["_is_fitted"]
|
| 1433 |
+
return obj
|
| 1434 |
+
|
| 1435 |
+
|
| 1436 |
+
# ===========================================================================
|
| 1437 |
+
# Utilities
|
| 1438 |
+
# ===========================================================================
|
| 1439 |
+
|
| 1440 |
+
def _is_jsonable(v: Any) -> bool:
|
| 1441 |
+
"""Check if a value is JSON serialisable."""
|
| 1442 |
+
try:
|
| 1443 |
+
json.dumps(v)
|
| 1444 |
+
return True
|
| 1445 |
+
except (TypeError, OverflowError, ValueError):
|
| 1446 |
+
return False
|
| 1447 |
+
|
| 1448 |
+
|
| 1449 |
+
# ===========================================================================
|
| 1450 |
+
# Model Registry — maps names to classes for the genetic algorithm
|
| 1451 |
+
# ===========================================================================
|
| 1452 |
+
|
| 1453 |
+
NEURAL_MODEL_REGISTRY: Dict[str, type] = {
|
| 1454 |
+
"lstm": LSTMSequenceModel,
|
| 1455 |
+
"transformer": TransformerAttentionModel,
|
| 1456 |
+
"tabnet": TabNetModel,
|
| 1457 |
+
"ft_transformer": FTTransformerModel,
|
| 1458 |
+
"deep_ensemble": DeepEnsemble,
|
| 1459 |
+
"conformal": ConformalPredictionWrapper,
|
| 1460 |
+
"autogluon": AutoGluonEnsemble,
|
| 1461 |
+
}
|
| 1462 |
+
|
| 1463 |
+
|
| 1464 |
+
def build_neural_model(model_type: str, **params) -> BaseNBAModel:
|
| 1465 |
+
"""
|
| 1466 |
+
Factory function to build a neural model by name.
|
| 1467 |
+
|
| 1468 |
+
Usage:
|
| 1469 |
+
model = build_neural_model("ft_transformer", n_tokens=128, d_token=64)
|
| 1470 |
+
model.fit(X_train, y_train)
|
| 1471 |
+
probs = model.predict_proba(X_test)
|
| 1472 |
+
|
| 1473 |
+
For conformal wrapper, pass base_model_type and base_model_params:
|
| 1474 |
+
model = build_neural_model(
|
| 1475 |
+
"conformal",
|
| 1476 |
+
base_model_type="deep_ensemble",
|
| 1477 |
+
base_model_params={"n_members": 5},
|
| 1478 |
+
alpha=0.1,
|
| 1479 |
+
)
|
| 1480 |
+
"""
|
| 1481 |
+
if model_type == "conformal":
|
| 1482 |
+
base_type = params.pop("base_model_type", "deep_ensemble")
|
| 1483 |
+
base_params = params.pop("base_model_params", {})
|
| 1484 |
+
base_model = build_neural_model(base_type, **base_params)
|
| 1485 |
+
return ConformalPredictionWrapper(base_model=base_model, **params)
|
| 1486 |
+
|
| 1487 |
+
cls = NEURAL_MODEL_REGISTRY.get(model_type)
|
| 1488 |
+
if cls is None:
|
| 1489 |
+
raise ValueError(
|
| 1490 |
+
f"Unknown model type '{model_type}'. "
|
| 1491 |
+
f"Available: {list(NEURAL_MODEL_REGISTRY.keys())}"
|
| 1492 |
+
)
|
| 1493 |
+
return cls(**params)
|
| 1494 |
+
|
| 1495 |
+
|
| 1496 |
+
# ===========================================================================
|
| 1497 |
+
# Quick smoke test (runs if executed directly)
|
| 1498 |
+
# ===========================================================================
|
| 1499 |
+
|
| 1500 |
+
if __name__ == "__main__":
|
| 1501 |
+
print("=" * 60)
|
| 1502 |
+
print("NBA Quant AI — Neural Models Smoke Test")
|
| 1503 |
+
print("=" * 60)
|
| 1504 |
+
|
| 1505 |
+
np.random.seed(42)
|
| 1506 |
+
N_TRAIN, N_TEST, N_FEAT = 500, 100, 200
|
| 1507 |
+
|
| 1508 |
+
X_train = np.random.randn(N_TRAIN, N_FEAT).astype(np.float32)
|
| 1509 |
+
# Inject some NaNs to test imputation
|
| 1510 |
+
mask = np.random.random(X_train.shape) < 0.05
|
| 1511 |
+
X_train[mask] = np.nan
|
| 1512 |
+
y_train = (np.random.random(N_TRAIN) > 0.5).astype(np.float32)
|
| 1513 |
+
|
| 1514 |
+
X_test = np.random.randn(N_TEST, N_FEAT).astype(np.float32)
|
| 1515 |
+
y_test = (np.random.random(N_TEST) > 0.5).astype(np.float32)
|
| 1516 |
+
|
| 1517 |
+
# Test each model (with small configs for speed)
|
| 1518 |
+
tests = [
|
| 1519 |
+
("FT-Transformer", FTTransformerModel(
|
| 1520 |
+
n_tokens=32, d_token=16, n_heads=2, n_layers=1,
|
| 1521 |
+
epochs=5, patience=3, batch_size=128,
|
| 1522 |
+
)),
|
| 1523 |
+
("Deep Ensemble (3 members)", DeepEnsemble(
|
| 1524 |
+
n_members=3, hidden_dims=(64, 32),
|
| 1525 |
+
epochs=5, patience=3, batch_size=128,
|
| 1526 |
+
)),
|
| 1527 |
+
("LSTM Sequence", LSTMSequenceModel(
|
| 1528 |
+
seq_len=5, hidden1=32, hidden2=16, dense_dim=16,
|
| 1529 |
+
epochs=5, patience=3, batch_size=128,
|
| 1530 |
+
)),
|
| 1531 |
+
("Transformer Attention", TransformerAttentionModel(
|
| 1532 |
+
seq_len=5, d_model=32, n_heads=2, n_layers=1,
|
| 1533 |
+
dim_ff=64, epochs=5, patience=3, batch_size=128,
|
| 1534 |
+
)),
|
| 1535 |
+
]
|
| 1536 |
+
|
| 1537 |
+
for name, model in tests:
|
| 1538 |
+
print(f"\n--- {name} ---")
|
| 1539 |
+
try:
|
| 1540 |
+
model.fit(X_train, y_train)
|
| 1541 |
+
probs = model.predict_proba(X_test)
|
| 1542 |
+
print(f" Predictions shape: {probs.shape}")
|
| 1543 |
+
print(f" Mean pred: {probs.mean():.4f}, Std: {probs.std():.4f}")
|
| 1544 |
+
print(f" Min: {probs.min():.4f}, Max: {probs.max():.4f}")
|
| 1545 |
+
print(f" Params: {list(model.get_params().keys())}")
|
| 1546 |
+
except Exception as e:
|
| 1547 |
+
print(f" ERROR: {e}")
|
| 1548 |
+
|
| 1549 |
+
# Test conformal wrapper
|
| 1550 |
+
print("\n--- Conformal Prediction Wrapper ---")
|
| 1551 |
+
try:
|
| 1552 |
+
base = DeepEnsemble(
|
| 1553 |
+
n_members=2, hidden_dims=(64, 32),
|
| 1554 |
+
epochs=5, patience=3, batch_size=128,
|
| 1555 |
+
)
|
| 1556 |
+
conformal = ConformalPredictionWrapper(base_model=base, alpha=0.1)
|
| 1557 |
+
conformal.fit(X_train, y_train)
|
| 1558 |
+
probs = conformal.predict_proba(X_test)
|
| 1559 |
+
sets = conformal.predict_sets(X_test)
|
| 1560 |
+
lower, upper = conformal.predict_intervals(X_test)
|
| 1561 |
+
print(f" Point preds shape: {probs.shape}")
|
| 1562 |
+
print(f" Prediction sets (first 5): {sets[:5]}")
|
| 1563 |
+
print(f" Intervals: [{lower[:3]}] - [{upper[:3]}]")
|
| 1564 |
+
print(f" Avg interval width: {(upper - lower).mean():.4f}")
|
| 1565 |
+
except Exception as e:
|
| 1566 |
+
print(f" ERROR: {e}")
|
| 1567 |
+
|
| 1568 |
+
# Test TabNet (may fail if pytorch_tabnet not installed)
|
| 1569 |
+
print("\n--- TabNet ---")
|
| 1570 |
+
try:
|
| 1571 |
+
tab = TabNetModel(
|
| 1572 |
+
n_d=8, n_a=8, n_steps=3, epochs=5, patience=3, batch_size=128,
|
| 1573 |
+
)
|
| 1574 |
+
tab.fit(X_train, y_train)
|
| 1575 |
+
probs = tab.predict_proba(X_test)
|
| 1576 |
+
print(f" Predictions shape: {probs.shape}")
|
| 1577 |
+
print(f" Mean pred: {probs.mean():.4f}")
|
| 1578 |
+
fi = tab.get_feature_importances()
|
| 1579 |
+
if fi is not None:
|
| 1580 |
+
print(f" Feature importances shape: {fi.shape}")
|
| 1581 |
+
except ImportError:
|
| 1582 |
+
print(" SKIPPED (pytorch_tabnet not installed)")
|
| 1583 |
+
except Exception as e:
|
| 1584 |
+
print(f" ERROR: {e}")
|
| 1585 |
+
|
| 1586 |
+
# Test factory
|
| 1587 |
+
print("\n--- Factory: build_neural_model ---")
|
| 1588 |
+
try:
|
| 1589 |
+
m = build_neural_model("ft_transformer", n_tokens=32, d_token=16,
|
| 1590 |
+
n_heads=2, n_layers=1, epochs=3, batch_size=128)
|
| 1591 |
+
m.fit(X_train, y_train)
|
| 1592 |
+
print(f" Factory FT-Transformer OK, preds mean: {m.predict_proba(X_test).mean():.4f}")
|
| 1593 |
+
except Exception as e:
|
| 1594 |
+
print(f" ERROR: {e}")
|
| 1595 |
+
|
| 1596 |
+
print("\n" + "=" * 60)
|
| 1597 |
+
print("Smoke test complete.")
|
| 1598 |
+
print("=" * 60)
|
requirements.txt
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
numpy>=2.0
|
| 2 |
+
scikit-learn>=1.5
|
| 3 |
+
xgboost>=3.0
|
| 4 |
+
lightgbm>=4.0
|
| 5 |
+
nba_api>=1.4
|
| 6 |
+
gradio>=5.0
|
| 7 |
+
uvicorn>=0.30
|
| 8 |
+
catboost>=1.2
|
| 9 |
+
psycopg2-binary>=2.9
|
| 10 |
+
# --- Neural network models (2025-2026 SOTA) ---
|
| 11 |
+
torch>=2.3 --index-url https://download.pytorch.org/whl/cpu
|
| 12 |
+
pytorch_tabnet>=4.1
|
| 13 |
+
mapie>=0.9
|
| 14 |
+
betacal>=0.1
|
| 15 |
+
# autogluon.tabular>=1.2 # OPTIONAL — large install (~2GB), uncomment if needed
|
| 16 |
+
# --- Browser scraping (needs Playwright deps in Docker image, see Dockerfile.browser) ---
|
| 17 |
+
# crawl4ai>=0.4 # OPTIONAL — uncomment when using Dockerfile.browser for browser-based scraping
|
| 18 |
+
html2text>=2024.2 # Lightweight HTML-to-markdown for requests fallback
|
| 19 |
+
beautifulsoup4>=4.12 # CSS selector extraction in requests fallback
|