own-solver/README.md

NeuroGolf Solver

Builds minimal ONNX networks for ARC-AGI tasks. Modular Python package with opset 10-17, zero-cost Slice-based transforms + embedded trained weights.

Public LB: 5331 (V97, 386/400 tasks solved by our solver)

Target: LB > 6043

submission-6043.zip already scored 6043 on Kaggle. It's in this repo. The models inside it are NOT generated by our solver — they come from an external source. To beat 6043 with OUR solver, we need to reverse-engineer those models into DSL solver code.

---

⚠️ MANDATORY: Use neurogolf_utils.py for ALL validation and scoring

own-solver/neurogolf_utils.py is the official Kaggle scoring kernel. It is the ONLY source of truth.

Any script that validates models or measures cost MUST use it. Do NOT write your own validator. Do NOT write your own cost estimator. Do NOT silently remove it when it crashes — debug it and fix the calling code instead.

What it provides:

• sanitize_model(model) — rename tensors to safe names (Kaggle does this before scoring)
• score_network(sanitized, trace_path) — returns (memory, params) using ORT Profiler
• verify_network(network, task_num, examples) — full validation + scoring
• convert_to_numpy(example) — encode grid to one-hot tensor
• run_network(session, input) — run inference with > 0.0 thresholding
• calculate_memory(model, trace_path) — ORT Profiler memory measurement
• calculate_params(model) — official param counting

How to use it for validation:

from neurogolf_utils import sanitize_model, score_network, convert_to_numpy, run_network
import onnxruntime

# 1. Load and sanitize
model = onnx.load(path)
sanitized = sanitize_model(model)

# 2. Create session with profiling
options = onnxruntime.SessionOptions()
options.enable_profiling = True
options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
session = onnxruntime.InferenceSession(sanitized.SerializeToString(), options)

# 3. Validate against examples
for example in all_examples:
    benchmark = convert_to_numpy(example)
    if benchmark is None: continue
    output = run_network(session, benchmark["input"])
    if not np.array_equal(output, benchmark["output"]):
        # FAILED

# 4. Get score
trace_path = session.end_profiling()
memory, params = score_network(sanitized, trace_path)
cost = memory + params
score = max(1.0, 25.0 - math.log(max(1.0, cost)))

Rules for agents:

1. NEVER write your own validate_model() function — use neurogolf_utils.py
2. NEVER write your own measure_cost() function — use score_network()
3. NEVER silently catch errors from neurogolf_utils.py — log them with full traceback
4. NEVER remove ORT profiler and replace with "static estimation" — static estimation is WRONG (proven: gives 5879 when Kaggle gives 5322)
5. If neurogolf_utils.py crashes, the bug is in YOUR calling code, not in the utils

---

Scoring Formula (neurogolf_utils.py, May 14)

score = max(1.0, 25.0 - math.log(max(1.0, memory + params)))

• memory = ORT Profiler intermediate tensor bytes (excludes input/output)
• params = initializer element count + Constant element count
• Initializer BYTES are NOT in memory — only element counts go to params
• NO MACs (removed May 4)

What We Have

Source	LB Score	Origin
submission-6043.zip	6043	External. Not from our solver. Needs reverse-engineering.
submission-5743.zip	5743	External. Partially reverse-engineered into wave22.
submission-v90.zip	5322	Our solver output (V90).
V97 (solver only)	5331	Our solver with wave23.
V5 (merged)	5748	6043 base + 5743 overrides (partial validation).

Pipeline

Solver only (generates models from scratch):

cd own-solver
python -m neurogolf_solver.main \
  --kaggle \
  --data_dir /kaggle/input/competitions/neurogolf-2026 \
  --output_dir /kaggle/working/submission \
  --fallback_zip fallback_unsolved.zip \
  --conv_budget 5.0

Merge (use 6043 as base, override with validated cheaper models):

cd own-solver
python merge_safe.py \
  --sub_6043 ../submission-6043.zip \
  --sub_5743 ../submission-5743.zip \
  --data_dir /kaggle/input/competitions/neurogolf-2026 \
  --output_zip /kaggle/working/merged.zip

Key Files

File	Purpose
neurogolf_utils.py	OFFICIAL Kaggle scoring kernel. USE THIS FOR ALL VALIDATION AND SCORING.
neurogolf_solver/	Our solver package (generates ONNX models from task data)
merge_safe.py	Safe merger using neurogolf_utils.py for validation + scoring
merge_best_of_both.py	Old merger (DO NOT USE — uses broken static cost estimation)
stage1_trivial_optimizer.py	Rebuilds trivial tasks with cheaper architecture
sign_corrected_conv.py	Fits direct Conv→output weights (3/18 working)
V90_IMPROVEMENT_PLAN.md	Documented analysis of improvement paths

Repo

https://huggingface.co/rogermt/neurogolf-solver