diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..039aa9b54d45d7562993a5a7f535e96c9b52ab25 100644 --- a/.gitattributes +++ b/.gitattributes @@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text +assets/fig2.png filter=lfs diff=lfs merge=lfs -text diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000000000000000000000000000000000000..896be7e1bc28dfad7eab19cb6320fb3e3a025a35 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,28 @@ +# syntax=docker/dockerfile:1.7 +# Linux/amd64 descriptor resolved from the official Python registry on 2026-07-28. +ARG BASE_IMAGE=python:3.12-slim@sha256:cab2dbf575e971934a81e4622f5aba17aa7929719bd7e31033a3a83b97fd0464 +FROM ${BASE_IMAGE} + +ARG SCIENCE_SPEC_SHA256=c0ded12138cb7c15a15966294ed9bdb87d3a66d2373a7d90050578dbb973c6fd +LABEL org.opencontainers.image.title="neutral-sheaf-admm-reproduction" +LABEL org.opencontainers.image.licenses="Apache-2.0" +LABEL org.opencontainers.image.revision="1e2b5d648361802234348b0b1a7fb3a222128e7d" +LABEL reproduction.science-spec-sha256="${SCIENCE_SPEC_SHA256}" + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 \ + JAX_DEFAULT_MATMUL_PRECISION=highest \ + TRACE_MODE=none \ + TRACKIO_LOGBOOK_AUTONOTE=0 + +WORKDIR /repro +COPY pyproject.toml uv.lock LICENSE SCIENCE-SPEC.yaml ./ +RUN test "$(sha256sum SCIENCE-SPEC.yaml | cut -d' ' -f1)" = "${SCIENCE_SPEC_SHA256}" \ + && python -m pip install --no-cache-dir uv==0.11.9 \ + && uv sync --frozen --no-dev +COPY configs ./configs +COPY src ./src +COPY scripts ./scripts +ENV PATH="/repro/.venv/bin:${PATH}" PYTHONPATH="/repro/src" +EXPOSE 7860 +CMD ["python", "-m", "http.server", "7860"] diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..74b79ffe5039e7a486b70b468c24a66df94478d0 --- /dev/null +++ b/LICENSE @@ -0,0 +1,161 @@ +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and +distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the +copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other +entities that control, are controlled by, or are under common control with +that entity. For the purposes of this definition, "control" means (i) the +power, direct or indirect, to cause the direction or management of such +entity, whether by contract or otherwise, or (ii) ownership of fifty percent +(50%) or more of the outstanding shares, or (iii) beneficial ownership of +such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising +permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, +including but not limited to software source code, documentation source, and +configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or +translation of a Source form, including but not limited to compiled object +code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, +made available under the License, as indicated by a copyright notice that is +included in or attached to the work. + +"Derivative Works" shall mean any work, whether in Source or Object form, +that is based on (or derived from) the Work and for which the editorial +revisions, annotations, elaborations, or other modifications represent, as a +whole, an original work of authorship. For the purposes of this License, +Derivative Works shall not include works that remain separable from, or merely +link (or bind by name) to the interfaces of, the Work and Derivative Works +thereof. + +"Contribution" shall mean any work of authorship, including the original +version of the Work and any modifications or additions to that Work or +Derivative Works thereof, that is intentionally submitted to Licensor for +inclusion in the Work by the copyright owner or by an individual or Legal +Entity authorized to submit on behalf of the copyright owner. For the purposes +of this definition, "submitted" means any form of electronic, verbal, or +written communication sent to the Licensor or its representatives, including +but not limited to communication on electronic mailing lists, source code +control systems, and issue tracking systems that are managed by, or on behalf +of, the Licensor for the purpose of discussing and improving the Work, but +excluding communication that is conspicuously marked or otherwise designated +in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf +of whom a Contribution has been received by Licensor and subsequently +incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this +License, each Contributor hereby grants to You a perpetual, worldwide, +non-exclusive, no-charge, royalty-free, irrevocable copyright license to +reproduce, prepare Derivative Works of, publicly display, publicly perform, +sublicense, and distribute the Work and such Derivative Works in Source or +Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this +License, each Contributor hereby grants to You a perpetual, worldwide, +non-exclusive, no-charge, royalty-free, irrevocable patent license to make, +have made, use, offer to sell, sell, import, and otherwise transfer the Work, +where such license applies only to those patent claims licensable by such +Contributor that are necessarily infringed by their Contribution(s) alone or +by combination of their Contribution(s) with the Work to which such +Contribution(s) was submitted. If You institute patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging that the +Work or a Contribution incorporated within the Work constitutes direct or +contributory patent infringement, then any patent licenses granted to You +under this License for that Work shall terminate as of the date such +litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or +Derivative Works thereof in any medium, with or without modifications, and in +Source or Object form, provided that You meet the following conditions: + +(a) You must give any other recipients of the Work or Derivative Works a copy +of this License; and + +(b) You must cause any modified files to carry prominent notices stating that +You changed the files; and + +(c) You must retain, in the Source form of any Derivative Works that You +distribute, all copyright, patent, trademark, and attribution notices from the +Source form of the Work, excluding those notices that do not pertain to any +part of the Derivative Works; and + +(d) If the Work includes a "NOTICE" text file as part of its distribution, then +any Derivative Works that You distribute must include a readable copy of the +attribution notices contained within such NOTICE file, excluding those notices +that do not pertain to any part of the Derivative Works, in at least one of +the following places: within a NOTICE text file distributed as part of the +Derivative Works; within the Source form or documentation, if provided along +with the Derivative Works; or within a display generated by the Derivative +Works, if and wherever such third-party notices normally appear. The contents +of the NOTICE file are for informational purposes only and do not modify the +License. You may add Your own attribution notices within Derivative Works that +You distribute, alongside or as an addendum to the NOTICE text from the Work, +provided that such additional attribution notices cannot be construed as +modifying the License. + +You may add Your own copyright statement to Your modifications and may provide +additional or different license terms and conditions for use, reproduction, or +distribution of Your modifications, or for any such Derivative Works as a +whole, provided Your use, reproduction, and distribution of the Work otherwise +complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any +Contribution intentionally submitted for inclusion in the Work by You to the +Licensor shall be under the terms and conditions of this License, without any +additional terms or conditions. Notwithstanding the above, nothing herein shall +supersede or modify the terms of any separate license agreement you may have +executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, +trademarks, service marks, or product names of the Licensor, except as required +for reasonable and customary use in describing the origin of the Work and +reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in +writing, Licensor provides the Work (and each Contributor provides its +Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied, including, without limitation, any warranties +or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A +PARTICULAR PURPOSE. You are solely responsible for determining the +appropriateness of using or redistributing the Work and assume any risks +associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in +tort (including negligence), contract, or otherwise, unless required by +applicable law (such as deliberate and grossly negligent acts) or agreed to in +writing, shall any Contributor be liable to You for damages, including any +direct, indirect, special, incidental, or consequential damages of any character +arising as a result of this License or out of the use or inability to use the +Work (including but not limited to damages for loss of goodwill, work stoppage, +computer failure or malfunction, or any and all other commercial damages or +losses), even if such Contributor has been advised of the possibility of such +damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or +Derivative Works thereof, You may choose to offer, and charge a fee for, +acceptance of support, warranty, indemnity, or other liability obligations +and/or rights consistent with this License. However, in accepting such +obligations, You may act only on Your own behalf and on Your sole +responsibility, not on behalf of any other Contributor, and only if You agree +to indemnify, defend, and hold each Contributor harmless for any liability +incurred by, or claims asserted against, such Contributor by reason of your +accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS diff --git a/README.md b/README.md index 3345b1f8427300f8a77ce62118d290bd04a815aa..6e25e6c4eba58ec99e466390f9c64a7705bcde50 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,115 @@ --- -title: Sheaf Admm Icml2026 Executor -emoji: 🌖 +title: Sheaf-ADMM ICML 2026 Reproduction Executor +emoji: 🧩 colorFrom: indigo colorTo: blue sdk: docker +app_port: 7860 pinned: false +license: apache-2.0 --- -Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference +# Sheaf-ADMM + +![Sheaf-ADMM overview](assets/fig2.png) + +Official JAX/Flax implementation of **Learning Multi-Agent Coordination via +Sheaf-ADMM** (ICML 2026). + +[![arXiv](https://img.shields.io/badge/arXiv-2605.31005-b31b1b?style=flat-square)](https://arxiv.org/abs/2605.31005) +[![Blog](https://img.shields.io/badge/Blog-Sakana%20AI-1f6feb?style=flat-square)](https://pub.sakana.ai/sheaf-admm/) + +Sheaf-ADMM decomposes an input into overlapping local views, each processed by an +agent that solves a small convex subproblem parameterized by a neural encoder. +Agents coordinate through the Alternating Direction Method of Multipliers (ADMM), +with the inter-agent constraints specified by a *cellular sheaf* — which aspects +of neighboring solutions must agree. The optimization is unrolled for a fixed +number of iterations, so the whole pipeline is differentiable and every component +is trained end-to-end. + +## Installation + +This repository is intended to be run from a source checkout. Install +[uv](https://docs.astral.sh/uv/getting-started/installation/), then run: + +```bash +uv sync +``` + +For CUDA experiments, install a JAX build matching your driver: + +```bash +uv pip install -U "jax[cuda12]" +``` + +## Data + +```bash +uv run python -m sheaf_admm.data.build_maze \ + --height 19 \ + --width 19 \ + --train-size 10000 \ + --test-size 1000 \ + --min-path-length 18 \ + --ood-sizes \ + --output-dir datasets/maze_std3_19px_10k + +uv run python -m sheaf_admm.data.build_mnist \ + --output-dir datasets/mnist + +uv run python -m sheaf_admm.data.build_sudoku \ + --output-dir datasets/sudoku_easy +``` + +The MNIST and Sudoku builders download their source datasets on first run. + +## Training + +Each task has a Sheaf-ADMM model and a recurrent-MPNN baseline: + +```bash +# Maze +uv run python scripts/train.py +experiment=maze_sheaf +uv run python scripts/train.py +experiment=maze_mpnn + +# MNIST +uv run python scripts/train.py +experiment=mnist_sheaf +uv run python scripts/train.py +experiment=mnist_mpnn + +# Sudoku +uv run python scripts/train.py +experiment=sudoku_sheaf +uv run python scripts/train.py +experiment=sudoku_sheaf_lora +uv run python scripts/train.py +experiment=sudoku_mpnn +``` + +Set `training.seed=42`, `123`, or `456` for the paper seeds. Set +`wandb.mode=online` to enable Weights & Biases logging. Checkpoints and +`history.json` are written to Hydra's run directory under `outputs/`. + +## Visualization + +The visualization script expects a Sheaf-ADMM checkpoint: + +```bash +uv run python -m scripts.visualize \ + --checkpoint outputs//