own-solver/README.md

# NeuroGolf Solver

Builds minimal ONNX networks for ARC-AGI tasks. Modular Python package with opset 10-17, zero-cost Slice-based transforms + embedded trained weights.

**Public LB: 5331** (V97, 386/400 tasks solved by our solver)

## Target: LB > 6043

submission-6043.zip already scored 6043 on Kaggle. It's in this repo.
The models inside it are NOT generated by our solver — they come from an external source.
To beat 6043 with OUR solver, we need to reverse-engineer those models into DSL solver code.

---

## ⚠️ MANDATORY: Use neurogolf_utils.py for ALL validation and scoring

**`own-solver/neurogolf_utils.py` is the official Kaggle scoring kernel. It is the ONLY source of truth.**

Any script that validates models or measures cost MUST use it. Do NOT write your own validator. Do NOT write your own cost estimator. Do NOT silently remove it when it crashes — debug it and fix the calling code instead.

### What it provides:
- `sanitize_model(model)` — rename tensors to safe names (Kaggle does this before scoring)
- `score_network(sanitized, trace_path)` — returns `(memory, params)` using ORT Profiler
- `verify_network(network, task_num, examples)` — full validation + scoring
- `convert_to_numpy(example)` — encode grid to one-hot tensor
- `run_network(session, input)` — run inference with `> 0.0` thresholding
- `calculate_memory(model, trace_path)` — ORT Profiler memory measurement
- `calculate_params(model)` — official param counting

### How to use it for validation:
```python
from neurogolf_utils import sanitize_model, score_network, convert_to_numpy, run_network
import onnxruntime

# 1. Load and sanitize
model = onnx.load(path)
sanitized = sanitize_model(model)

# 2. Create session with profiling
options = onnxruntime.SessionOptions()
options.enable_profiling = True
options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
session = onnxruntime.InferenceSession(sanitized.SerializeToString(), options)

# 3. Validate against examples
for example in all_examples:
    benchmark = convert_to_numpy(example)
    if benchmark is None: continue
    output = run_network(session, benchmark["input"])
    if not np.array_equal(output, benchmark["output"]):
        # FAILED

# 4. Get score
trace_path = session.end_profiling()
memory, params = score_network(sanitized, trace_path)
cost = memory + params
score = max(1.0, 25.0 - math.log(max(1.0, cost)))
```

### Rules for agents:
1. **NEVER** write your own `validate_model()` function — use `neurogolf_utils.py`
2. **NEVER** write your own `measure_cost()` function — use `score_network()`
3. **NEVER** silently catch errors from `neurogolf_utils.py` — log them with full traceback
4. **NEVER** remove ORT profiler and replace with "static estimation" — static estimation is WRONG (proven: gives 5879 when Kaggle gives 5322)
5. If `neurogolf_utils.py` crashes, the bug is in YOUR calling code, not in the utils

---

## Scoring Formula (neurogolf_utils.py, May 14)

```python
score = max(1.0, 25.0 - math.log(max(1.0, memory + params)))
```
- memory = ORT Profiler intermediate tensor bytes (excludes input/output)
- params = initializer element count + Constant element count
- Initializer BYTES are NOT in memory — only element counts go to params
- NO MACs (removed May 4)

## What We Have

| Source | LB Score | Origin |
|--------|----------|--------|
| submission-6043.zip | 6043 | External. Not from our solver. Needs reverse-engineering. |
| submission-5743.zip | 5743 | External. Partially reverse-engineered into wave22. |
| submission-v90.zip | 5322 | Our solver output (V90). |
| V97 (solver only) | 5331 | Our solver with wave23. |
| V5 (merged) | 5748 | 6043 base + 5743 overrides (partial validation). |

## Pipeline

### Solver only (generates models from scratch):
```bash
cd own-solver
python -m neurogolf_solver.main \
  --kaggle \
  --data_dir /kaggle/input/competitions/neurogolf-2026 \
  --output_dir /kaggle/working/submission \
  --fallback_zip fallback_unsolved.zip \
  --conv_budget 5.0
```

### Merge (use 6043 as base, override with validated cheaper models):
```bash
cd own-solver
python merge_safe.py \
  --sub_6043 ../submission-6043.zip \
  --sub_5743 ../submission-5743.zip \
  --data_dir /kaggle/input/competitions/neurogolf-2026 \
  --output_zip /kaggle/working/merged.zip
```

## Key Files

| File | Purpose |
|------|---------|
| `neurogolf_utils.py` | **OFFICIAL Kaggle scoring kernel. USE THIS FOR ALL VALIDATION AND SCORING.** |
| `neurogolf_solver/` | Our solver package (generates ONNX models from task data) |
| `merge_safe.py` | Safe merger using neurogolf_utils.py for validation + scoring |
| `merge_best_of_both.py` | Old merger (DO NOT USE — uses broken static cost estimation) |
| `stage1_trivial_optimizer.py` | Rebuilds trivial tasks with cheaper architecture |
| `sign_corrected_conv.py` | Fits direct Conv→output weights (3/18 working) |
| `V90_IMPROVEMENT_PLAN.md` | Documented analysis of improvement paths |

## Repo

https://huggingface.co/rogermt/neurogolf-solver