A missing LFS object looks fine from this machine
Browse filesThe two tester images added for the README exist here, are LFS-tracked here,
and are referenced by a correct media.githubusercontent URL -- and both return
404, because their objects have never left this disk. Nothing local says so.
The tree is right, the pointer is right, the working copy is the real PNG.
`tools/check_lfs_urls.py` fetches every image the README and the Icon/Banner
fields point at and asserts three things per URL: it comes back 200, it does
not begin with a pointer header, and it is the same byte count as the file on
disk. Status alone is not enough -- raw.githubusercontent.com serves the
pointer as `200 OK` / `text/plain`, which is the silent half of this trap and
the reason the size comparison is there. Falsified against the live server on
both branches: the two tester URLs fail on status, and a raw. URL for a file
that is otherwise fine returns 130 bytes beginning `version https://git-lfs`,
which trips the other two assertions.
It is the only checker that leaves the machine, so it stays out of
check_all.py; the docs say to run it after the push and before publishing.
CLAUDE.md also gains the ordering constraint the 404s exposed: `git lfs push
<url> <branch>` covers only the objects reachable from that branch, so a
picture added on a feature branch is not covered by a `main` push until the
merge has landed. Merge, then objects, then branch.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PLXmbwfdXirQ5oFreXPcMi
- CLAUDE.md +8 -1
- tools/check_lfs_urls.py +132 -0
|
@@ -33,10 +33,17 @@ git lfs push https://github.com/dntpi/ComfyUI-Hand-Tie-Clips.git main
|
|
| 33 |
git push origin main
|
| 34 |
```
|
| 35 |
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
- **`raw.githubusercontent.com` does not resolve an LFS pointer.** It serves the 131-byte pointer file as `200 OK` / `text/plain`, so every `<img>` breaks with no error anywhere. Use `media.githubusercontent.com/media/<owner>/<repo>/<ref>/<path>`, which is what the README and the `Icon`/`Banner` fields point at. `github.com/<o>/<r>/raw/` behaves like `raw.`, not like `media.`.
|
| 39 |
- **HuggingFace refuses plain binaries regardless of size** -- it rejected PNGs of 262-545 KB, not just the >10 MB the old `.gitattributes` comment assumed. Anything binary under `docs/img/` must be LFS or the mirror push fails.
|
|
|
|
| 40 |
|
| 41 |
## Architecture
|
| 42 |
|
|
|
|
| 33 |
git push origin main
|
| 34 |
```
|
| 35 |
|
| 36 |
+
**`git lfs push <url> <branch>` uploads only the objects reachable from that
|
| 37 |
+
branch**, so a picture added on a feature branch is not covered by a `main`
|
| 38 |
+
push until the merge has landed locally. Merge first, then push the objects,
|
| 39 |
+
then push the branch. Getting that order wrong is how two images sat on `v2`
|
| 40 |
+
for a release cycle while the `main` push reported success.
|
| 41 |
+
|
| 42 |
+
Three further traps behind that one, all silent:
|
| 43 |
|
| 44 |
- **`raw.githubusercontent.com` does not resolve an LFS pointer.** It serves the 131-byte pointer file as `200 OK` / `text/plain`, so every `<img>` breaks with no error anywhere. Use `media.githubusercontent.com/media/<owner>/<repo>/<ref>/<path>`, which is what the README and the `Icon`/`Banner` fields point at. `github.com/<o>/<r>/raw/` behaves like `raw.`, not like `media.`.
|
| 45 |
- **HuggingFace refuses plain binaries regardless of size** -- it rejected PNGs of 262-545 KB, not just the >10 MB the old `.gitattributes` comment assumed. Anything binary under `docs/img/` must be LFS or the mirror push fails.
|
| 46 |
+
- **A missing object is not visible from this machine.** The tree, the pointer and the working copy are all correct locally whether or not the upload happened; only a fetch tells you. `tools/check_lfs_urls.py` fetches every image the README and the `Icon`/`Banner` fields reference and asserts each comes back 200, not starting with a pointer header, and the same byte count as the file on disk -- which is what catches the `raw.` case, since that one is a successful response. It is the only checker that needs the network, so it is not in `check_all.py`: run it after the push and before `comfy node publish`.
|
| 47 |
|
| 48 |
## Architecture
|
| 49 |
|
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Every image the published page and the registry listing point at, fetched.
|
| 2 |
+
|
| 3 |
+
This is the one check that has to leave the machine, which is why it is not in
|
| 4 |
+
check_all.py: run it after the push, before `comfy node publish`.
|
| 5 |
+
|
| 6 |
+
Two failures it exists for, both silent:
|
| 7 |
+
|
| 8 |
+
1. **An LFS object that was never uploaded.** `origin` carries two push URLs
|
| 9 |
+
and git-lfs uploads to the first only, so objects can be missing on GitHub
|
| 10 |
+
while the tree that references them is not. `git lfs push <github-url>
|
| 11 |
+
<branch>` fixes it -- but only for objects reachable from THAT branch, so
|
| 12 |
+
a picture added on a feature branch needs the merge to land first. Get the
|
| 13 |
+
order wrong and the page renders a broken image with no error anywhere.
|
| 14 |
+
|
| 15 |
+
2. **A pointer served as 200 OK.** `raw.githubusercontent.com` does not
|
| 16 |
+
resolve LFS: it returns the 131-byte pointer file, as text/plain, with a
|
| 17 |
+
success status. Nothing in a browser, a linter or a link checker calls
|
| 18 |
+
that an error. Only the size gives it away.
|
| 19 |
+
|
| 20 |
+
So status is not enough. Each URL must come back the same number of bytes as
|
| 21 |
+
the file on disk, and must not begin with a pointer header.
|
| 22 |
+
"""
|
| 23 |
+
from __future__ import annotations
|
| 24 |
+
|
| 25 |
+
import io
|
| 26 |
+
import os
|
| 27 |
+
import re
|
| 28 |
+
import subprocess
|
| 29 |
+
import sys
|
| 30 |
+
import urllib.error
|
| 31 |
+
import urllib.request
|
| 32 |
+
|
| 33 |
+
HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 34 |
+
|
| 35 |
+
# The pointer file's first line. Assembled, so a hit on this file's own text
|
| 36 |
+
# cannot be mistaken for the thing it looks for.
|
| 37 |
+
POINTER = "version " + "https://git-lfs"
|
| 38 |
+
|
| 39 |
+
MEDIA = "https://media.githubusercontent.com/media/"
|
| 40 |
+
FAILS = []
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def ck(label, ok, detail=""):
|
| 44 |
+
print(" %-4s %-52s %s" % ("ok" if ok else "FAIL", label, detail))
|
| 45 |
+
if not ok:
|
| 46 |
+
FAILS.append(label)
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def urls_referenced():
|
| 50 |
+
"""Every remote image the two published faces of this pack point at."""
|
| 51 |
+
out = []
|
| 52 |
+
readme = io.open(os.path.join(HERE, "README.md"), encoding="utf-8").read()
|
| 53 |
+
for m in re.finditer(r"!\[[^\]]*\]\((https?://[^)]+)\)", readme):
|
| 54 |
+
out.append(("README.md", m.group(1)))
|
| 55 |
+
toml = io.open(os.path.join(HERE, "pyproject.toml"), encoding="utf-8").read()
|
| 56 |
+
for field in ("Icon", "Banner"):
|
| 57 |
+
m = re.search(r"^%s\s*=\s*\"([^\"]+)\"" % field, toml, re.M)
|
| 58 |
+
if m:
|
| 59 |
+
out.append(("pyproject.toml %s" % field, m.group(1)))
|
| 60 |
+
return out
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def local_path(url):
|
| 64 |
+
"""docs/img/x.png, out of .../media/<owner>/<repo>/<ref>/docs/img/x.png."""
|
| 65 |
+
if not url.startswith(MEDIA):
|
| 66 |
+
return None
|
| 67 |
+
rest = url[len(MEDIA):].split("/")
|
| 68 |
+
return "/".join(rest[3:]) if len(rest) > 3 else None
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def main():
|
| 72 |
+
refs = urls_referenced()
|
| 73 |
+
print("checking %d referenced image(s)\n" % len(refs))
|
| 74 |
+
|
| 75 |
+
tracked = subprocess.run(["git", "lfs", "ls-files", "-n"], cwd=HERE,
|
| 76 |
+
capture_output=True, text=True).stdout.split("\n")
|
| 77 |
+
tracked = {p.strip() for p in tracked if p.strip()}
|
| 78 |
+
|
| 79 |
+
for where, url in refs:
|
| 80 |
+
name = url.rsplit("/", 1)[-1]
|
| 81 |
+
# The whole point of media. is that raw. lies about LFS.
|
| 82 |
+
if "raw.githubusercontent.com" in url or "/raw/" in url:
|
| 83 |
+
ck("%s: not a raw. URL" % name, False,
|
| 84 |
+
"%s -- raw. serves the pointer as 200 OK" % where)
|
| 85 |
+
continue
|
| 86 |
+
|
| 87 |
+
rel = local_path(url)
|
| 88 |
+
if rel is None:
|
| 89 |
+
ck("%s: recognised host" % name, True, "not LFS, skipped")
|
| 90 |
+
continue
|
| 91 |
+
|
| 92 |
+
disk = os.path.join(HERE, rel.replace("/", os.sep))
|
| 93 |
+
if not os.path.exists(disk):
|
| 94 |
+
ck("%s: exists in this tree" % name, False, rel)
|
| 95 |
+
continue
|
| 96 |
+
size = os.path.getsize(disk)
|
| 97 |
+
|
| 98 |
+
# An image referenced from the page but not LFS-tracked pushes fine to
|
| 99 |
+
# GitHub and is refused by the HuggingFace mirror, which takes no plain
|
| 100 |
+
# binaries at any size.
|
| 101 |
+
ck("%s: LFS-tracked" % name, rel in tracked, rel)
|
| 102 |
+
|
| 103 |
+
try:
|
| 104 |
+
with urllib.request.urlopen(url, timeout=30) as r:
|
| 105 |
+
body = r.read()
|
| 106 |
+
status = r.status
|
| 107 |
+
except urllib.error.HTTPError as e:
|
| 108 |
+
ck("%s: served" % name, False,
|
| 109 |
+
"HTTP %d -- the object is not on GitHub; `git lfs push "
|
| 110 |
+
"<github-url> <branch>` for the branch that has it" % e.code)
|
| 111 |
+
continue
|
| 112 |
+
except OSError as e:
|
| 113 |
+
ck("%s: served" % name, False, "unreachable: %s" % e)
|
| 114 |
+
continue
|
| 115 |
+
|
| 116 |
+
ck("%s: served" % name, status == 200, "HTTP %d" % status)
|
| 117 |
+
ck("%s: content, not a pointer" % name,
|
| 118 |
+
not body[:40].decode("utf-8", "replace").startswith(POINTER),
|
| 119 |
+
"%d bytes" % len(body))
|
| 120 |
+
ck("%s: matches the file on disk" % name, len(body) == size,
|
| 121 |
+
"%d served vs %d on disk" % (len(body), size))
|
| 122 |
+
|
| 123 |
+
if FAILS:
|
| 124 |
+
print("\nLFS URL CHECK: %d failure(s). The page renders these broken."
|
| 125 |
+
% len(FAILS))
|
| 126 |
+
return 1
|
| 127 |
+
print("\nLFS URL CHECK: all passed")
|
| 128 |
+
return 0
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
if __name__ == "__main__":
|
| 132 |
+
sys.exit(main())
|