Update README.md: MANDATORY use of neurogolf_utils.py for all validation and scoring"

own-solver/README.md

@@ -2,7 +2,7 @@
 
 Builds minimal ONNX networks for ARC-AGI tasks. Modular Python package with opset 10-17, zero-cost Slice-based transforms + embedded trained weights.
 
-**Public LB: 5560.70** (V3, 400/400 tasks)
+**Public LB: 5331** (V97, 386/400 tasks solved by our solver)
 
 ## Target: LB > 6043
 
@@ -10,6 +10,62 @@ submission-6043.zip already scored 6043 on Kaggle. It's in this repo.
 The models inside it are NOT generated by our solver — they come from an external source.
 To beat 6043 with OUR solver, we need to reverse-engineer those models into DSL solver code.
 
+---
+
+## ⚠️ MANDATORY: Use neurogolf_utils.py for ALL validation and scoring
+
+**`own-solver/neurogolf_utils.py` is the official Kaggle scoring kernel. It is the ONLY source of truth.**
+
+Any script that validates models or measures cost MUST use it. Do NOT write your own validator. Do NOT write your own cost estimator. Do NOT silently remove it when it crashes — debug it and fix the calling code instead.
+
+### What it provides:
+- `sanitize_model(model)` — rename tensors to safe names (Kaggle does this before scoring)
+- `score_network(sanitized, trace_path)` — returns `(memory, params)` using ORT Profiler
+- `verify_network(network, task_num, examples)` — full validation + scoring
+- `convert_to_numpy(example)` — encode grid to one-hot tensor
+- `run_network(session, input)` — run inference with `> 0.0` thresholding
+- `calculate_memory(model, trace_path)` — ORT Profiler memory measurement
+- `calculate_params(model)` — official param counting
+
+### How to use it for validation:
+```python
+from neurogolf_utils import sanitize_model, score_network, convert_to_numpy, run_network
+import onnxruntime
+
+# 1. Load and sanitize
+model = onnx.load(path)
+sanitized = sanitize_model(model)
+
+# 2. Create session with profiling
+options = onnxruntime.SessionOptions()
+options.enable_profiling = True
+options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
+session = onnxruntime.InferenceSession(sanitized.SerializeToString(), options)
+
+# 3. Validate against examples
+for example in all_examples:
+    benchmark = convert_to_numpy(example)
+    if benchmark is None: continue
+    output = run_network(session, benchmark["input"])
+    if not np.array_equal(output, benchmark["output"]):
+        # FAILED
+
+# 4. Get score
+trace_path = session.end_profiling()
+memory, params = score_network(sanitized, trace_path)
+cost = memory + params
+score = max(1.0, 25.0 - math.log(max(1.0, cost)))
+```
+
+### Rules for agents:
+1. **NEVER** write your own `validate_model()` function — use `neurogolf_utils.py`
+2. **NEVER** write your own `measure_cost()` function — use `score_network()`
+3. **NEVER** silently catch errors from `neurogolf_utils.py` — log them with full traceback
+4. **NEVER** remove ORT profiler and replace with "static estimation" — static estimation is WRONG (proven: gives 5879 when Kaggle gives 5322)
+5. If `neurogolf_utils.py` crashes, the bug is in YOUR calling code, not in the utils
+
+---
+
 ## Scoring Formula (neurogolf_utils.py, May 14)
 
 ```python
@@ -26,27 +82,44 @@ score = max(1.0, 25.0 - math.log(max(1.0, memory + params)))
 |--------|----------|--------|
 | submission-6043.zip | 6043 | External. Not from our solver. Needs reverse-engineering. |
 | submission-5743.zip | 5743 | External. Partially reverse-engineered into wave22. |
-| submission-v90.zip | 5322 | Our solver output. |
-| V3 (merged) | 5560 | merge_best_of_both.py + stage1_trivial_optimizer.py |
+| submission-v90.zip | 5322 | Our solver output (V90). |
+| V97 (solver only) | 5331 | Our solver with wave23. |
+| V5 (merged) | 5748 | 6043 base + 5743 overrides (partial validation). |
 
-## Pipeline (current, working)
+## Pipeline
 
+### Solver only (generates models from scratch):
 ```bash
-python merge_best_of_both.py \
-  --sub_5743 submission-5743.zip \
-  --sub_v90 submission-v90.zip \
+cd own-solver
+python -m neurogolf_solver.main \
+  --kaggle \
   --data_dir /kaggle/input/competitions/neurogolf-2026 \
-  --output_zip /kaggle/working/merged.zip
+  --output_dir /kaggle/working/submission \
+  --fallback_zip fallback_unsolved.zip \
+  --conv_budget 5.0
+```
 
-python stage1_trivial_optimizer.py \
-  --input_zip /kaggle/working/merged.zip \
+### Merge (use 6043 as base, override with validated cheaper models):
+```bash
+cd own-solver
+python merge_safe.py \
+  --sub_6043 ../submission-6043.zip \
+  --sub_5743 ../submission-5743.zip \
   --data_dir /kaggle/input/competitions/neurogolf-2026 \
-  --output_zip /kaggle/working/submission.zip
+  --output_zip /kaggle/working/merged.zip
 ```
 
-## What Needs to Happen
+## Key Files
 
-Reverse-engineer submission-6043.zip models into solver code. See NEXT_AGENT.md.
+| File | Purpose |
+|------|---------|
+| `neurogolf_utils.py` | **OFFICIAL Kaggle scoring kernel. USE THIS FOR ALL VALIDATION AND SCORING.** |
+| `neurogolf_solver/` | Our solver package (generates ONNX models from task data) |
+| `merge_safe.py` | Safe merger using neurogolf_utils.py for validation + scoring |
+| `merge_best_of_both.py` | Old merger (DO NOT USE — uses broken static cost estimation) |
+| `stage1_trivial_optimizer.py` | Rebuilds trivial tasks with cheaper architecture |
+| `sign_corrected_conv.py` | Fits direct Conv→output weights (3/18 working) |
+| `V90_IMPROVEMENT_PLAN.md` | Documented analysis of improvement paths |
 
 ## Repo