# ⚠️ READ THIS FIRST — medal-solvers/LEARNING.md

## YOU MUST READ ALL 4 DOCS BEFORE DOING ANYTHING:
1. **LEARNING.md** (this file) — what works, what doesn't, scoring rules
2. **TODO.md** — the ORDERED task list. DO THE TOP ITEM. Don't invent your own.
3. **NEXT_AGENT.md** — current state, what was tried, what to do next
4. **BULK_SCAN.md** — profiled targets with base scores

## IF YOU HIT A WALL: Update these docs with WHAT you tried, WHY it failed, and WHAT you need. Do NOT silently move on.

---

## COMPETITIVE LANDSCAPE

| Metric | Value |
|--------|-------|
| **Current LB** | **6072.74 (v70)** |
| **Rank** | ~150 |
| **Target** | 6500 (silver) / 6100 (bronze) |
| **Top 10** | 7500+ (~4 pts/model across many tasks) |
| **Gap to silver** | ~427 pts |

---

## SCORING FORMULA

```python
score = max(1.0, 25.0 - math.log(max(1.0, memory + params)))
# memory = sum of (num_elements × bytes_per_element) for ALL value_info tensors
#          EXCEPT those named "input" or "output"
#          MAX across all example runs (from profiler trace)
# params = sum of num_elements for ALL initializers (element COUNT, not bytes!)
```

## ⚠️ CRITICAL: BULK_SCAN ESTIMATES ARE UNRELIABLE

**ALWAYS profile with score_model.py before building ONNX.**
The static estimates in BULK_SCAN.md can be wildly wrong:
- Task 314: BULK_SCAN estimated base ~12, actual profiled = **15.6**
- Task 057: Estimated gain, actual = **-0.011** (WORSE than base)
- Task 384: Estimated gain, actual = **-0.611** (WORSE than base)

**Both task057 and task384 were REMOVED from submission in v70 because they hurt the score.**

## TWO-PHASE OPTIMIZATION (PROVEN)

### Phase 1: Hand-craft ONNX model (biggest gains)
- Crack the task rule (need 265+/265 accuracy)
- Build compact ONNX with coordinate arithmetic
- Slice-first pattern: work in HxW space, pad to 30x30 at end

### Phase 2: Run onnxsim on the model (free +0.01 to +0.77 per model)
```python
from onnxsim import simplify
model_simp, ok = simplify(model)
del model_simp.graph.value_info[:]
model_simp = onnx.shape_inference.infer_shapes(model_simp, strict_mode=True)
onnx.save(model_simp, output_path)
```
**This is mandatory for EVERY model before upload.** Proven +1.46 total on v65.

## MANDATORY VERIFICATION BEFORE UPLOAD

```python
# 1. Validate correctness (265+/265)
# 2. Score YOUR model with FULL PROFILING
my_score = score_model(my_path, task_num, '/path/to/task-data/')
# 3. Score the BASE model with FULL PROFILING
base_score = score_model(base_path, task_num, '/path/to/task-data/')
# 4. ONLY upload if my_score > base_score
```

**NEVER upload without profiled scoring vs base.** Static estimates are WRONG.

## MANDATORY BUILD STEPS (every model)

```python
# ALWAYS before saving:
del model.graph.value_info[:]
model = onnx.shape_inference.infer_shapes(model, strict_mode=True)
# THEN run onnxsim:
from onnxsim import simplify
model, ok = simplify(model)
del model.graph.value_info[:]
model = onnx.shape_inference.infer_shapes(model, strict_mode=True)
onnx.save(model, output_path)
```

## SLICE-FIRST PATTERN (key optimization)

For fixed grids H×W < 30×30:
```
1. Slice input: [1,10,30,30] → [1,10,H,W]
2. Compute in H×W space (all intermediates small)
3. Pad back to [1,10,30,30] as last node outputting directly to 'output'
```

## WHAT MAKES A GOOD TARGET

1. Fixed grid size (small = fewer bytes per intermediate)
2. **PROFILED** base score < 12 AND large base model file (>50KB)
3. Rule expressible as **coordinate arithmetic** (row/col grids, modular ops, masks)
4. NO CC detection, NO dynamic pattern matching, NO rotation search
5. **VERIFY base score with score_model.py BEFORE spending time on ONNX build**

## VERIFIED BASE SCORES (profiled, not estimated)

| Task | Base Score | Base File | Potential Gain | Status |
|------|-----------|-----------|---------------|--------|
| 366 | 7.08 | 1.3MB | +8-10 | Rule not cracked |
| 219 | 8.42 | 87KB | +6 | Solver at 245/265 |
| 157 | 9.14 | 858KB | +5-6 | Rule partially understood |
| 76 | 10.34 | 949KB | +5-7 | Solver at 202/266 |
| 182 | 11.45 | 70KB | +3 | Needs CC detection |
| 363 | 11.78 | 170KB | +3 | 264/265 max |

## PROVEN FAILURES (do NOT retry)

| Approach | Result |
|----------|--------|
| Float16 conversion | Cast overhead cancels savings |
| onnx-neural-compressor quantization | Introduces DynamicQuantizeLinear/MatMulInteger → scorer returns None |
| Uploading without PROFILED scoring vs base | task057 and task384 scored WORSE → removed in v70 |
| Task 363 heuristics (×7) | 264/265 max, unknown tiebreaker |
| Tasks needing CC detection (101, 233, 191, 044, 077, 182) | Can't express in ONNX efficiently |
| Base score estimates from node count / BULK_SCAN | WRONG — must profile with score_model.py |
| onnxsim on base models | Only 3 out of 400 improved (209, 367, 323) — already done |
| Task 314 hand-craft ONNX | 266/266 correct BUT scored 14.46 vs base 15.62 = WORSE |
| Simple gravity/color-replace/flip scans | All viable tasks have tiny base models (< 3KB), can't beat |
| Scanning all 400 tasks for simple transforms | Only found 3x3 grid tasks with base 16-25, unbeatable |

## WHAT WORKS

| Approach | Result |
|----------|--------|
| Hand-craft ONNX + onnxsim | +30 pts across 12 models (v49→v66) |
| onnxsim on existing hand-crafted models | +1.46 free points (v64→v65) |
| onnxsim on base models with >5% size reduction | +0.46 (tasks 209, 367, 323) |
| Gather-based coordinate indexing | Works for variable-position patterns |
| Gather-based per-column shift (task 128) | 20 nodes, 2KB, +0.155 |
| ConvTranspose for pixel-repeat upscale | Avoids Resize op issues |
| Conv stride-N for block summation | Efficient macro-cell detection |
| Removing models that score worse than base | +0.622 instant (task057, task384 removed) |

## TOOLS

| Tool | Command |
|------|---------|
| Score model | `python score_model.py --model X.onnx --task-num N --task-data-dir task-data/` |
| Simplify | `pip install onnxsim` then use in Python |
| Base models | In `submission-6043.zip` (400 models) |
| Task data | In `own-solver/neurogolf-2026.zip` |
| ARC DSL | `michaelhodel-arc-dsl.txt` (primitives for cracking rules) |