rogermt commited on
Commit
05d80c8
Β·
verified Β·
1 Parent(s): 57eaa81

Update README.md to reflect V65 state (LB 1553.13, 113 tasks)

Browse files
Files changed (1) hide show
  1. own-solver/README.md +47 -39
own-solver/README.md CHANGED
@@ -1,56 +1,64 @@
1
- # NeuroGolf Solver v5.3
2
 
3
  Builds minimal ONNX networks for ARC-AGI tasks. Modular Python package with opset 17, zero-cost Slice-based transforms.
4
 
5
- **Public LB: 1313.61** (V46, 97/400 solved, commit `38521bc`)
6
 
7
- ## Session Summary (2026-05-16)
8
 
9
- **LB 1071.38 β†’ 1313.61 (+242.23 pts, +22.6%)**
10
 
11
- Tasks solved: 80 β†’ 97 (+17 new analytical tasks)
12
 
13
- ### What Worked
14
 
15
- | Solver | Tasks | Technique |
16
- |--------|-------|-----------|
17
- | `s_row_uniform_fill` | 52 | 3Γ—3 col equality β†’ color 5 |
18
- | `s_majority_replace_5` | 229 | ArgMax channel counts β†’ majority mask |
19
- | `s_find_color_with_hole` | 291 | Per-channel bbox deficit β†’ ArgMax |
20
- | `s_fill_bbox_per_color` | 132 | CumSum fwd+bwd range fill |
21
- | `s_border_detect` | 98 | Pad+Slice erosion |
22
- | `s_symmetry_check_lr` | 103 | Gather(reverse) β†’ Equal β†’ ReduceMin |
23
- | `s_variable_scale_2x` | 307 | ConvTranspose(stride=2, group=10) |
24
- | wave3 bg fix | 1, 304 | grid_mask β†’ bg_oh β†’ Add in self_tile |
25
- | `s_split_midpoint_or` | 347 | Split halves, OR β†’ fixed color |
26
- | `s_split_nor` | 26,144,227,386,395 | Split halves, NOR β†’ fixed color |
27
- | `s_split_boolean` | 318 | Split with separator, OR β†’ fixed color |
28
- | `s_symmetrize` | (287 local, not on Kaggle) | Copy half to mirror |
29
 
30
- ### What Failed (removed from registry)
31
 
32
- | Solver | Issue |
33
- |--------|-------|
34
- | `s_count_to_row` (Task 339) | Passes 100% locally, 0 pts on Kaggle |
35
- | `s_variable_tile_1x2` (Task 249) | Same β€” correct locally, rejected by Kaggle |
 
36
 
37
- ### Unsolved Task Analysis
38
 
39
- - 303 tasks remain unsolved
40
- - **110 superset** (fill operations) β€” most need non-linear/global rules
41
- - **35 same-positions** (color remaps) β€” position-dependent, not simple color_map
42
- - **53 completely different** β€” complex transformations
43
- - **100 diff-shape** β€” mostly crop/extract operations needing object detection
44
- - Conv solver already tries kernel sizes 1-29 with PCR fallback
45
- - Simple patterns (flip, rotate, tile, color_map, split, quadrant) all exhausted
46
- - Remaining tasks need: multi-step composition, object detection, or program synthesis
47
 
48
- ### Key Facts
49
 
50
- - Scoring: `points = max(1.0, 25.0 - log(max(1.0, memory + params)))` (MACs don't count)
51
- - Top scores 7500+ β†’ they solve 300+ tasks (likely program synthesis approaches)
52
- - Conv solver ceiling: ~36 tasks regardless of budget
53
- - `neurogolf_utils.py` in repo is V10 (outdated), Kaggle uses V11
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
  ## Repo
56
 
 
1
+ # NeuroGolf Solver v5.5
2
 
3
  Builds minimal ONNX networks for ARC-AGI tasks. Modular Python package with opset 17, zero-cost Slice-based transforms.
4
 
5
+ **Public LB: 1553.13** (V65, 113/400 solved, commit `57eaa81`)
6
 
7
+ ## Session Summary (2026-05-19)
8
 
9
+ **LB 1430.24 β†’ 1553.13 (+122.89 pts, +8.6%)**
10
 
11
+ Tasks solved: 105 β†’ 113 (+8 new tasks via reverse-engineering submission-5743)
12
 
13
+ ### What Worked β€” Wave 6+7: Reverse-Engineering 5743
14
 
15
+ | Solver | Task | Score | Technique |
16
+ |--------|------|-------|-----------|
17
+ | `s_periodic_tile_fill` | 033 | 13.64 | Period-6 Gather tiling + bg-fill from last-row color |
18
+ | `s_bbox_crop_3x3` | 039 | 14.27 | ReduceSum→ArgMax→Gather(3 consecutive) |
19
+ | `s_centroid_zoom` | 065 | 13.53 | Single-pixel centroid β†’ shift indices β†’ Gather |
20
+ | `s_eightfold_symmetry` | 074 | 14.36 | D4 symmetry: Max of 8 flip/shift/transpose |
21
+ | `s_ushape_fill_bottom` | 126 | 13.29 | 5-directional shift multiply β†’ U-detect β†’ fill color 4 |
22
+ | `s_select_asymmetric_block` | 146 | 15.64 | Transpose-Equal symmetry test on 3 stacked blocks |
23
+ | `s_marker_grid_fill` | 043 | 15.26 | Row 0 + col 9 markers β†’ fill intersections with color 2 |
24
+ | `s_histogram_match` | 399 | 14.16 | L1 histogram distance to bank β†’ select closest output |
 
 
 
 
25
 
26
+ ### What Failed
27
 
28
+ | Attempt | Issue |
29
+ |---------|-------|
30
+ | Two-layer ConvNet (`conv_twolayer.py`) | Trains too slow for 5s budget; 0 new tasks on Kaggle; adds 1200s runtime |
31
+ | Task 167 (`s_count_present_colors`) | Bank values don't match actual task data |
32
+ | Task 186 (`s_count_color1_lookup`) | Same β€” hardcoded bank doesn't match Kaggle data |
33
 
34
+ ### Key Methodology
35
 
36
+ 1. Download `submission-5743.zip` (LB leader's 400 ONNX models)
37
+ 2. Analyze each model's ONNX graph (nodes, initializers, architecture)
38
+ 3. Run with synthetic inputs to understand the transformation
39
+ 4. Build equivalent ONNX model with **minimal detection** (just build it, let `validate()` filter)
40
+ 5. Upload and submit β€” validate() catches mismatches on arc-gen
 
 
 
41
 
42
+ **Critical rule**: NEVER use strict Python detection logic for task-specific solvers. Build the model unconditionally, `validate()` is the gatekeeper.
43
 
44
+ ### Architecture Patterns Found in 5743
45
+
46
+ | Pattern | Count | Description |
47
+ |---------|-------|-------------|
48
+ | Neighbor counting (all-ones 3Γ—3 Conv) | 29 tasks | Sum neighborhood β†’ threshold β†’ action |
49
+ | Shift Conv (directional) | 50 tasks | Move content by 1px β†’ detect adjacency |
50
+ | Channel select + spatial pool | 31 tasks | Conv1x1(filter) + Conv(spatial) + threshold |
51
+ | Two-layer trained ConvNet | 6 tasks | Conv(K×K)→ReLU→Conv(1×1), gradient-descent weights |
52
+ | Lookup/classification | 20+ tasks | Count/histogram β†’ index into output bank |
53
+
54
+ ### File Structure
55
+
56
+ ```
57
+ neurogolf_solver/solvers/
58
+ wave6.py β€” 6 solvers (Tasks 33, 39, 65, 74, 126, 146)
59
+ wave7.py β€” 4 solvers (Tasks 43, 167, 186, 399) [2 working, 2 need correct banks]
60
+ conv_twolayer.py β€” PyTorch 2-layer ConvNet (not in registry, needs offline pre-training)
61
+ ```
62
 
63
  ## Repo
64