Sandpies Claude Opus 5 commited on
Commit
4416a19
·
1 Parent(s): 155eb60

A missing LFS object looks fine from this machine

Browse files

The two tester images added for the README exist here, are LFS-tracked here,
and are referenced by a correct media.githubusercontent URL -- and both return
404, because their objects have never left this disk. Nothing local says so.
The tree is right, the pointer is right, the working copy is the real PNG.

`tools/check_lfs_urls.py` fetches every image the README and the Icon/Banner
fields point at and asserts three things per URL: it comes back 200, it does
not begin with a pointer header, and it is the same byte count as the file on
disk. Status alone is not enough -- raw.githubusercontent.com serves the
pointer as `200 OK` / `text/plain`, which is the silent half of this trap and
the reason the size comparison is there. Falsified against the live server on
both branches: the two tester URLs fail on status, and a raw. URL for a file
that is otherwise fine returns 130 bytes beginning `version https://git-lfs`,
which trips the other two assertions.

It is the only checker that leaves the machine, so it stays out of
check_all.py; the docs say to run it after the push and before publishing.

CLAUDE.md also gains the ordering constraint the 404s exposed: `git lfs push
<url> <branch>` covers only the objects reachable from that branch, so a
picture added on a feature branch is not covered by a `main` push until the
merge has landed. Merge, then objects, then branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PLXmbwfdXirQ5oFreXPcMi

Files changed (2) hide show
  1. CLAUDE.md +8 -1
  2. tools/check_lfs_urls.py +132 -0
CLAUDE.md CHANGED
@@ -33,10 +33,17 @@ git lfs push https://github.com/dntpi/ComfyUI-Hand-Tie-Clips.git main
33
  git push origin main
34
  ```
35
 
36
- Two further traps behind that one, both silent:
 
 
 
 
 
 
37
 
38
  - **`raw.githubusercontent.com` does not resolve an LFS pointer.** It serves the 131-byte pointer file as `200 OK` / `text/plain`, so every `<img>` breaks with no error anywhere. Use `media.githubusercontent.com/media/<owner>/<repo>/<ref>/<path>`, which is what the README and the `Icon`/`Banner` fields point at. `github.com/<o>/<r>/raw/` behaves like `raw.`, not like `media.`.
39
  - **HuggingFace refuses plain binaries regardless of size** -- it rejected PNGs of 262-545 KB, not just the >10 MB the old `.gitattributes` comment assumed. Anything binary under `docs/img/` must be LFS or the mirror push fails.
 
40
 
41
  ## Architecture
42
 
 
33
  git push origin main
34
  ```
35
 
36
+ **`git lfs push <url> <branch>` uploads only the objects reachable from that
37
+ branch**, so a picture added on a feature branch is not covered by a `main`
38
+ push until the merge has landed locally. Merge first, then push the objects,
39
+ then push the branch. Getting that order wrong is how two images sat on `v2`
40
+ for a release cycle while the `main` push reported success.
41
+
42
+ Three further traps behind that one, all silent:
43
 
44
  - **`raw.githubusercontent.com` does not resolve an LFS pointer.** It serves the 131-byte pointer file as `200 OK` / `text/plain`, so every `<img>` breaks with no error anywhere. Use `media.githubusercontent.com/media/<owner>/<repo>/<ref>/<path>`, which is what the README and the `Icon`/`Banner` fields point at. `github.com/<o>/<r>/raw/` behaves like `raw.`, not like `media.`.
45
  - **HuggingFace refuses plain binaries regardless of size** -- it rejected PNGs of 262-545 KB, not just the >10 MB the old `.gitattributes` comment assumed. Anything binary under `docs/img/` must be LFS or the mirror push fails.
46
+ - **A missing object is not visible from this machine.** The tree, the pointer and the working copy are all correct locally whether or not the upload happened; only a fetch tells you. `tools/check_lfs_urls.py` fetches every image the README and the `Icon`/`Banner` fields reference and asserts each comes back 200, not starting with a pointer header, and the same byte count as the file on disk -- which is what catches the `raw.` case, since that one is a successful response. It is the only checker that needs the network, so it is not in `check_all.py`: run it after the push and before `comfy node publish`.
47
 
48
  ## Architecture
49
 
tools/check_lfs_urls.py ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Every image the published page and the registry listing point at, fetched.
2
+
3
+ This is the one check that has to leave the machine, which is why it is not in
4
+ check_all.py: run it after the push, before `comfy node publish`.
5
+
6
+ Two failures it exists for, both silent:
7
+
8
+ 1. **An LFS object that was never uploaded.** `origin` carries two push URLs
9
+ and git-lfs uploads to the first only, so objects can be missing on GitHub
10
+ while the tree that references them is not. `git lfs push <github-url>
11
+ <branch>` fixes it -- but only for objects reachable from THAT branch, so
12
+ a picture added on a feature branch needs the merge to land first. Get the
13
+ order wrong and the page renders a broken image with no error anywhere.
14
+
15
+ 2. **A pointer served as 200 OK.** `raw.githubusercontent.com` does not
16
+ resolve LFS: it returns the 131-byte pointer file, as text/plain, with a
17
+ success status. Nothing in a browser, a linter or a link checker calls
18
+ that an error. Only the size gives it away.
19
+
20
+ So status is not enough. Each URL must come back the same number of bytes as
21
+ the file on disk, and must not begin with a pointer header.
22
+ """
23
+ from __future__ import annotations
24
+
25
+ import io
26
+ import os
27
+ import re
28
+ import subprocess
29
+ import sys
30
+ import urllib.error
31
+ import urllib.request
32
+
33
+ HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
34
+
35
+ # The pointer file's first line. Assembled, so a hit on this file's own text
36
+ # cannot be mistaken for the thing it looks for.
37
+ POINTER = "version " + "https://git-lfs"
38
+
39
+ MEDIA = "https://media.githubusercontent.com/media/"
40
+ FAILS = []
41
+
42
+
43
+ def ck(label, ok, detail=""):
44
+ print(" %-4s %-52s %s" % ("ok" if ok else "FAIL", label, detail))
45
+ if not ok:
46
+ FAILS.append(label)
47
+
48
+
49
+ def urls_referenced():
50
+ """Every remote image the two published faces of this pack point at."""
51
+ out = []
52
+ readme = io.open(os.path.join(HERE, "README.md"), encoding="utf-8").read()
53
+ for m in re.finditer(r"!\[[^\]]*\]\((https?://[^)]+)\)", readme):
54
+ out.append(("README.md", m.group(1)))
55
+ toml = io.open(os.path.join(HERE, "pyproject.toml"), encoding="utf-8").read()
56
+ for field in ("Icon", "Banner"):
57
+ m = re.search(r"^%s\s*=\s*\"([^\"]+)\"" % field, toml, re.M)
58
+ if m:
59
+ out.append(("pyproject.toml %s" % field, m.group(1)))
60
+ return out
61
+
62
+
63
+ def local_path(url):
64
+ """docs/img/x.png, out of .../media/<owner>/<repo>/<ref>/docs/img/x.png."""
65
+ if not url.startswith(MEDIA):
66
+ return None
67
+ rest = url[len(MEDIA):].split("/")
68
+ return "/".join(rest[3:]) if len(rest) > 3 else None
69
+
70
+
71
+ def main():
72
+ refs = urls_referenced()
73
+ print("checking %d referenced image(s)\n" % len(refs))
74
+
75
+ tracked = subprocess.run(["git", "lfs", "ls-files", "-n"], cwd=HERE,
76
+ capture_output=True, text=True).stdout.split("\n")
77
+ tracked = {p.strip() for p in tracked if p.strip()}
78
+
79
+ for where, url in refs:
80
+ name = url.rsplit("/", 1)[-1]
81
+ # The whole point of media. is that raw. lies about LFS.
82
+ if "raw.githubusercontent.com" in url or "/raw/" in url:
83
+ ck("%s: not a raw. URL" % name, False,
84
+ "%s -- raw. serves the pointer as 200 OK" % where)
85
+ continue
86
+
87
+ rel = local_path(url)
88
+ if rel is None:
89
+ ck("%s: recognised host" % name, True, "not LFS, skipped")
90
+ continue
91
+
92
+ disk = os.path.join(HERE, rel.replace("/", os.sep))
93
+ if not os.path.exists(disk):
94
+ ck("%s: exists in this tree" % name, False, rel)
95
+ continue
96
+ size = os.path.getsize(disk)
97
+
98
+ # An image referenced from the page but not LFS-tracked pushes fine to
99
+ # GitHub and is refused by the HuggingFace mirror, which takes no plain
100
+ # binaries at any size.
101
+ ck("%s: LFS-tracked" % name, rel in tracked, rel)
102
+
103
+ try:
104
+ with urllib.request.urlopen(url, timeout=30) as r:
105
+ body = r.read()
106
+ status = r.status
107
+ except urllib.error.HTTPError as e:
108
+ ck("%s: served" % name, False,
109
+ "HTTP %d -- the object is not on GitHub; `git lfs push "
110
+ "<github-url> <branch>` for the branch that has it" % e.code)
111
+ continue
112
+ except OSError as e:
113
+ ck("%s: served" % name, False, "unreachable: %s" % e)
114
+ continue
115
+
116
+ ck("%s: served" % name, status == 200, "HTTP %d" % status)
117
+ ck("%s: content, not a pointer" % name,
118
+ not body[:40].decode("utf-8", "replace").startswith(POINTER),
119
+ "%d bytes" % len(body))
120
+ ck("%s: matches the file on disk" % name, len(body) == size,
121
+ "%d served vs %d on disk" % (len(body), size))
122
+
123
+ if FAILS:
124
+ print("\nLFS URL CHECK: %d failure(s). The page renders these broken."
125
+ % len(FAILS))
126
+ return 1
127
+ print("\nLFS URL CHECK: all passed")
128
+ return 0
129
+
130
+
131
+ if __name__ == "__main__":
132
+ sys.exit(main())