Sandpies Claude Opus 5 commited on
Commit
fadddaa
·
1 Parent(s): 3a25965

pin_renorm: a band-matched mode, because sigma is a no-op here

Browse files

The ratchet drifts the high-band FRACTION of the pin latent. A fraction
is scale-invariant, and a uniform rescale is all `pin_renorm=on` ever
did. Driven end to end with a 12.74% band drift planted in hop 2:
`sigma` applied x0.9651 and left the drift at +12.74%; `band` took it
to -0.04%. There is no gain or anchor choice that rescues a
scale-invariant statistic from a scale -- the lever was mis-specified,
not mis-tuned. Kept as `sigma` because pre-0.5 workflows saved "on".

pin_renorm is now ["off", "sigma", "band"]; "on" maps to "sigma", and
the combo keeps its widget slot.

Two defects caught on the way, both of the kind that look like success:

match_band first used `k = target * sigma / hi_sigma`, but scaling the
high band changes the sigma it is a fraction of, so the target moves as
you apply it -- it landed 5% short, in the right direction. It now
solves the orthogonal fixed point and refines against the measured
statistic.

The test fixture used torch.randn, whose high-band fraction is 0.966 --
against its ceiling, where the clamp does the correcting and every
assertion measures the clamp. Real latents sit at 0.3643; the fixture
now lands there and an assertion holds it in that regime.

Also: pin_cond was keyed into hop 1, which has no pin, so flipping a
lever re-rendered a byte-identical hop. A third of every A/B, on runs
already too slow to repeat.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015MGjcAV8bDy93qJfmLi9kw

Files changed (5) hide show
  1. CLAUDE.md +1 -0
  2. docs/DEVLOG.md +75 -0
  3. h3_ref_chain.py +110 -52
  4. latents.py +97 -0
  5. tools/check_texture.py +65 -0
CLAUDE.md CHANGED
@@ -364,6 +364,7 @@ hop N-1 ends. Hop N opens by holding what it was handed.
364
  - Fewer than 3 wired reference stills triggers a warning log; unconnected `Load Image` nodes do not count as wired.
365
  - **Never judge texture drift with mean `|Laplacian|`, or with any whole-frame scalar.** Measured on two 44 s H3 chains whose faces visibly come apart, it reads **0.961 and 0.973** -- one of them says the clip improved. It is an area average (a face is ~6% of a portrait frame and loses to the background sixteen to one) and it pools every spatial frequency, so band-to-band movement cancels. On those clips global contrast FELL while mid-band energy ROSE. Use `tools/texture_probe.py`: three Gaussian-difference bands (fine <1px / **mid 1-2.5px, the one that moves** / coarse 2.5-6px), a subject box against a background control, and the within-hop slope. `--video` runs it on any file, including other people's rigs. See DEVLOG 26.
366
  - **The texture ratchet is injected AT the join, and nowhere else.** Measured off the hop cache on a 3x243f chain, head box, mid band: within each hop last/first is `0.996 / 1.003 / 1.018` -- flat -- and each join steps `x1.042`, twice, identically. It is a staircase, not a ramp. So the hand-off is the cause and not merely the carrier, which is the good news: a hand-off-side lever can remove this rather than damp it. `prev_imgs = imgs[-tail_n:]` and the pin latent are conditioning-only copies (`.clone()`, and `_condition_pin_latent` rebuilds rather than mutates), so they can be corrected without touching a single visible pixel.
 
367
  - **Texture drift is independent of exposure, but the metric is not.** A band-pass removes the local mean, so it ignores a brightness *offset* -- it is not scale-free, and a 5% brighter picture measures ~5% more band energy with texture unchanged. `texture_probe` prints both (`x` raw, `n` exposure-normalised); where they disagree the difference is the scene brightening. On the 3-hop chain whole-frame mid read `x1.084` raw but `n1.035` normalised. Do not expect `tone_compensate` to touch the ratchet, and do not tune on a 2-hop chain; the effect only separates at 3+.
368
  - Soundtrack is official H3, not a pack dialect: `(S1)` + `<d>…</d>` for lines, `overall_soundscape` for ambience/physical (or `N/A` for requested silence). Do not invent beat-keywords. “No speech” / “no dialogue” remains negation/gibberish.
369
 
 
364
  - Fewer than 3 wired reference stills triggers a warning log; unconnected `Load Image` nodes do not count as wired.
365
  - **Never judge texture drift with mean `|Laplacian|`, or with any whole-frame scalar.** Measured on two 44 s H3 chains whose faces visibly come apart, it reads **0.961 and 0.973** -- one of them says the clip improved. It is an area average (a face is ~6% of a portrait frame and loses to the background sixteen to one) and it pools every spatial frequency, so band-to-band movement cancels. On those clips global contrast FELL while mid-band energy ROSE. Use `tools/texture_probe.py`: three Gaussian-difference bands (fine <1px / **mid 1-2.5px, the one that moves** / coarse 2.5-6px), a subject box against a background control, and the within-hop slope. `--video` runs it on any file, including other people's rigs. See DEVLOG 26.
366
  - **The texture ratchet is injected AT the join, and nowhere else.** Measured off the hop cache on a 3x243f chain, head box, mid band: within each hop last/first is `0.996 / 1.003 / 1.018` -- flat -- and each join steps `x1.042`, twice, identically. It is a staircase, not a ramp. So the hand-off is the cause and not merely the carrier, which is the good news: a hand-off-side lever can remove this rather than damp it. `prev_imgs = imgs[-tail_n:]` and the pin latent are conditioning-only copies (`.clone()`, and `_condition_pin_latent` rebuilds rather than mutates), so they can be corrected without touching a single visible pixel.
367
+ - **`pin_renorm=sigma` cannot fix the ratchet, and no tuning will rescue it.** The drifting statistic is the high-band *fraction* -- a ratio -- and a ratio is invariant under any uniform rescale, which is all a sigma match does. Measured end to end through the node: a 12.74% band drift stayed at **+12.74%** under `sigma` and went to **-0.04%** under `band`. Total sigma is not merely a weak proxy here; on the real chain it FELL 1.2% while the fraction rose, so it does not even point the right way. `sigma` is kept only because pre-0.5 workflows saved it as `on`.
368
  - **Texture drift is independent of exposure, but the metric is not.** A band-pass removes the local mean, so it ignores a brightness *offset* -- it is not scale-free, and a 5% brighter picture measures ~5% more band energy with texture unchanged. `texture_probe` prints both (`x` raw, `n` exposure-normalised); where they disagree the difference is the scene brightening. On the 3-hop chain whole-frame mid read `x1.084` raw but `n1.035` normalised. Do not expect `tone_compensate` to touch the ratchet, and do not tune on a 2-hop chain; the effect only separates at 3+.
369
  - Soundtrack is official H3, not a pack dialect: `(S1)` + `<d>…</d>` for lines, `overall_soundscape` for ambience/physical (or `N/A` for requested silence). Do not invent beat-keywords. “No speech” / “no dialogue” remains negation/gibberish.
370
 
docs/DEVLOG.md CHANGED
@@ -1211,3 +1211,78 @@ The 3-hop chain's luma rose 4.7%, so whole-frame mid read `x1.084` when the
1211
  texture part was `n1.035`. Both columns are printed now. The head box was
1212
  unaffected either way -- the brightening was in the background -- which is
1213
  exactly the kind of thing a single whole-frame number cannot tell you.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1211
  texture part was `n1.035`. Both columns are printed now. The head box was
1212
  unaffected either way -- the brightening was in the background -- which is
1213
  exactly the kind of thing a single whole-frame number cannot tell you.
1214
+
1215
+ ## 27. The band lever, and why the old one could never have worked (2026-09-01)
1216
+
1217
+ Built after §26's correction, on the finding that the ratchet is a step at the
1218
+ join: +4.2% mid-band, twice, identically, on a 3-hop chain. Two identical steps
1219
+ is already a model -- constant multiplicative step per join, geometric in hop
1220
+ count. It predicts hop 3 at 1.042^2 = 1.086 against 1.079 measured. So the
1221
+ shape did not need a 4-5 hop run to pin down, which matters: those runs are
1222
+ expensive enough that the user does not do them.
1223
+
1224
+ `pin_renorm` is now `["off", "sigma", "band"]`. `"on"` maps to `"sigma"`, so
1225
+ pre-0.5 workflows keep their behaviour, and the combo keeps its widget slot --
1226
+ adding options is safe, adding widgets is not.
1227
+
1228
+ ### The old lever is a no-op, provably
1229
+
1230
+ The statistic that drifts is the high-band **fraction**, hi_sigma / sigma. A
1231
+ fraction is invariant under uniform rescaling, and a uniform rescale is the
1232
+ entirety of what `sigma` mode does. Driven end to end through
1233
+ `_condition_pin_latent` with a 12.74% band drift planted in hop 2:
1234
+
1235
+ mode=off ratio after 0.3526 (anchor 0.3128) err +12.74%
1236
+ mode=sigma ratio after 0.3526 (anchor 0.3128) err +12.74% x0.9651 applied
1237
+ mode=band ratio after 0.3127 (anchor 0.3128) err -0.04% hi x0.8339
1238
+
1239
+ `sigma` applied a real scale factor and moved the drift by nothing at all. This
1240
+ is stronger than §26's "corrects the wrong way": there is no gain, no strength
1241
+ knob and no anchor choice that makes a scale-invariant statistic respond to a
1242
+ scale. The lever was mis-specified, not mis-tuned. It is kept only for the
1243
+ workflows that saved it.
1244
+
1245
+ ### The fixed point that nearly shipped
1246
+
1247
+ `match_band` first computed `k = target * sigma / hi_sigma`. That is wrong in a
1248
+ way that hides: scaling the high band changes the sigma it is a fraction of, so
1249
+ the target moves while you apply it. It landed at 0.3331 against a 0.3168
1250
+ target -- 5% short, in the right direction, which is the worst possible
1251
+ signature because it looks like it works.
1252
+
1253
+ Now it solves the orthogonal fixed point in closed form,
1254
+ `k = r*L / (H*sqrt(1-r^2))`, then refines two or three passes against the
1255
+ statistic as actually measured, because a difference of Gaussians is not an
1256
+ exact projection. Lands at 0.3167 against 0.3168.
1257
+
1258
+ ### The fixture was also wrong, and would have hidden it
1259
+
1260
+ The first test used `torch.randn` for the latent. White noise has a high-band
1261
+ fraction of **0.966** -- pinned against its ceiling of 1.0, where lifting the
1262
+ high band moves the statistic by 0.6% and the clamp does all the "correcting".
1263
+ Every assertion about the lever would have been measuring the clamp. Real
1264
+ latents sit at 0.3643, so the fixture is now built to land near there and an
1265
+ assertion holds it in that regime.
1266
+
1267
+ The safety property is asserted as "the entire change lies along the high band"
1268
+ (cosine with `hi` > 0.99), not as "the low band is unchanged" -- re-splitting
1269
+ the result does not hand back the same `lo`, because the split is not a
1270
+ projection. The first version asserted the false one and failed correctly.
1271
+
1272
+ ### One cache key narrowed
1273
+
1274
+ `pin_cond` was in every hop's key including hop 1, which has no pin --
1275
+ `_pin_mech_for` returns `"none"` at index 0 and the conditioning branch is
1276
+ `elif i > 0`. So flipping a lever discarded a byte-identical cached hop 1 and
1277
+ re-rendered it. That is a third of the cost of every lever A/B, on the one hop
1278
+ that provably could not have changed. Now keyed only from hop 2.
1279
+
1280
+ ### Still unknown
1281
+
1282
+ The latent's band fraction moved 1.6% across the chain while the picture's mid
1283
+ band moved 8%. The decode is nonlinear so they are not expected to be
1284
+ proportional, but a full match to hop 1's fraction may therefore under-correct
1285
+ the picture. That is one A/B to find out, and it is readable off a 3-hop run:
1286
+ `texture_probe` reports each join separately, so two joins is two data points.
1287
+ If `band` shrinks the +4.2% step but does not close it, the next move is a gain
1288
+ above 1.0, fitted -- not guessed.
h3_ref_chain.py CHANGED
@@ -628,77 +628,115 @@ def _rebuild_latent_samples(x, parts):
628
  return _latents.rebuild(x, parts)
629
 
630
 
631
- def _condition_pin_latent(lat, anchor_std, renorm=False, noise=0.0, seed=0):
632
  """Anti-ratchet preprocessing for the latent handed to Motion-Context.
633
 
634
  MiniMaxH3MotionContext.apply() takes `context_latent` as-is and exposes no
635
- hook, so both levers have to be applied to the latent before it goes in.
636
 
637
- `renorm` rescales this pin so its standard deviation matches hop 2's. The
638
- pin's own sigma climbs hop over hop, and because that inflated pin is what
639
- conditions the next hop the growth compounds upstream of anything a master
640
- pass can reach. A scalar rescale moves no structure, so it cannot blur
641
- detail. `noise` mixes in a seeded perturbation, which attacks the same
642
- ratchet from the other side; measured gains reverse above 0.10, hence the
643
- widget cap.
 
 
 
 
 
 
 
 
 
 
 
644
 
645
  **Per component, not per latent** (fixed 2026-08-27). Video and audio are
646
- two tensors in one NestedTensor and their sigmas drift independently, so
647
  each carries its own anchor. Before this, `.std()` raised on the nested
648
- object and both levers were dead -- the failure announced itself once per
649
- hop as `pin conditioning skipped` and was easy to read as routine noise.
650
- A single global scale would also have been wrong on its own terms, letting
651
- the much larger video component dictate the audio's correction.
652
-
653
- Returns `(latent, new_anchor_std)` where the anchor is a list, one sigma
654
- per component -- hop 2 establishes what hops 3+ are matched against. Both
655
- levers default off, in which case the latent is returned untouched.
656
  """
657
  if not isinstance(lat, dict) or "samples" not in lat:
658
- return lat, anchor_std
659
  x = lat["samples"]
660
  parts = _latent_parts(x)
661
  if parts is None:
662
  print(f"[{TAG}] pin conditioning skipped: unrecognised latent "
663
  f"({type(x).__name__})", flush=True)
664
- return lat, anchor_std
 
 
 
665
  try:
666
- cur = [float(t.float().std()) for t in parts]
 
 
 
667
  except Exception as e: # noqa: BLE001
668
  print(f"[{TAG}] pin conditioning skipped ({e!r})", flush=True)
669
- return lat, anchor_std
670
- if not all(c == c and c for c in cur): # zero or NaN in any stream
671
- return lat, anchor_std
672
- if anchor_std is None:
673
- anchor_std = cur
674
- if len(anchor_std) != len(cur):
675
  # Stream count changed mid-chain. Nothing sensible to match against.
676
  print(f"[{TAG}] pin conditioning skipped: latent has {len(cur)} "
677
- f"component(s), anchor has {len(anchor_std)}", flush=True)
678
- return lat, anchor_std
679
- if not renorm and noise <= 0.0:
680
- return lat, anchor_std
 
 
 
 
 
 
 
 
 
 
 
 
 
681
  out_parts, notes = [], []
682
- for idx, (t, c, a) in enumerate(zip(parts, cur, anchor_std)):
683
  o = t
684
- if renorm:
685
- scale = a / c
686
  o = o * scale
687
- notes.append(f"renorm[{idx}] x{scale:.4f} (sigma {c:.4f} -> {a:.4f})")
 
 
 
 
 
 
 
 
 
 
688
  if noise > 0.0:
689
  # Per component: `.shape` on the nested object reports only the
690
  # first component's shape, so one draw for the whole latent would
691
  # size its noise to the video and broadcast that onto the audio.
692
  g = torch.Generator(device="cpu").manual_seed((int(seed) + idx) & 0x7FFFFFFF)
693
  n = torch.randn(o.shape, generator=g, dtype=torch.float32)
694
- o = o + n.to(dtype=o.dtype, device=o.device) * (float(noise) * a)
695
  notes.append(f"noise[{idx}] {float(noise):.3f}")
696
  out_parts.append(o)
697
  if notes:
698
  print(f"[{TAG}] pin conditioning: " + ", ".join(notes), flush=True)
699
  new = dict(lat)
700
  new["samples"] = _rebuild_latent_samples(x, out_parts)
701
- return new, anchor_std
702
 
703
 
704
  def _pin_mech_for(hop_index, overlap_n, prev_sampled):
@@ -1124,15 +1162,24 @@ class HandTieClips:
1124
  "follows the overlap widget."
1125
  ),
1126
  }),
1127
- "pin_renorm": (["off", "on"], {
1128
  "default": "off",
1129
  "tooltip": (
1130
- "Rescale each pinned latent so its spread matches hop 1's. "
1131
- "The pin's own sigma climbs every hop and that inflated pin "
1132
- "conditions the next one, so texture ratchets up along a "
1133
- "long chain. A scalar rescale moves no structure, so it "
1134
- "cannot blur detail. Off reproduces chain_00038 exactly; "
1135
- "turn it on for chains of 3+ hops."
 
 
 
 
 
 
 
 
 
1136
  ),
1137
  }),
1138
  "pin_noise": ("FLOAT", {
@@ -1513,13 +1560,16 @@ class HandTieClips:
1513
  pbar = comfy.utils.ProgressBar(n)
1514
 
1515
  hop_store = None
1516
- pin_renorm_on = str(pin_renorm) == "on"
 
 
 
1517
  pin_noise_v = max(0.0, min(0.10, float(pin_noise)))
1518
  audio_ctx = int(audio_pin_frames) if int(audio_pin_frames) > 0 else int(overlap_n)
1519
- pin_anchor_std = None # hop 2's pin sets the sigma hops 3+ match
1520
- if pin_renorm_on or pin_noise_v > 0.0:
1521
  print(f"[{TAG}] pin conditioning enabled: "
1522
- f"renorm={'on' if pin_renorm_on else 'off'} "
1523
  f"noise={pin_noise_v:.3f}", flush=True)
1524
  tone_mode = str(tone_compensate)
1525
  tone_on = tone_mode != "off" and tone_mode in _tone.MODES
@@ -1753,7 +1803,15 @@ class HandTieClips:
1753
  # has no sampler latent and falls back to AddGuide, which
1754
  # is a different render of the same inputs.
1755
  "pin_mech": pin_mech_pred,
1756
- "pin_cond": (pin_renorm_on, round(pin_noise_v, 4), audio_ctx),
 
 
 
 
 
 
 
 
1757
  })
1758
  # A locked shot reuses its last render even though its inputs
1759
  # changed -- that is the point of locking. The content key would
@@ -1804,9 +1862,9 @@ class HandTieClips:
1804
  elif i > 0:
1805
  pin_latent = prev_sampled
1806
  if pin_latent is not None:
1807
- pin_latent, pin_anchor_std = _condition_pin_latent(
1808
- pin_latent, pin_anchor_std,
1809
- renorm=pin_renorm_on, noise=pin_noise_v,
1810
  seed=(int(seed) + i))
1811
  cond, pin_mech_used = _pin_continue(
1812
  cond, latent, vae, audio_vae, overlap_n,
 
628
  return _latents.rebuild(x, parts)
629
 
630
 
631
+ def _condition_pin_latent(lat, anchor, mode="off", noise=0.0, seed=0):
632
  """Anti-ratchet preprocessing for the latent handed to Motion-Context.
633
 
634
  MiniMaxH3MotionContext.apply() takes `context_latent` as-is and exposes no
635
+ hook, so every lever has to be applied to the latent before it goes in.
636
 
637
+ Two rescale modes, and the difference between them is the whole point.
638
+
639
+ `sigma` rescales the pin so its standard deviation matches the anchor hop's.
640
+ This is the original lever and **it is measurably the wrong statistic.** On
641
+ a 3-hop chain the pin's total sigma FELL (1.0414 -> 1.0289) while the
642
+ picture's mid-band energy climbed 8% and its high-band fraction rose 1.6%.
643
+ Matching sigma there scales the whole latent UP by 1.2%, lifting a high
644
+ band that was already too hot. Kept because it is what shipped, and old
645
+ workflows say "on".
646
+
647
+ `band` splits each spatial component into low and high and rescales only the
648
+ high part, so the *ratio* between them returns to the anchor hop's. That
649
+ ratio is what the ratchet actually moves. Still a scalar per band, so it
650
+ moves no structure and cannot blur or invent detail -- the property that
651
+ made `sigma` safe to run blind, kept.
652
+
653
+ `noise` mixes in a seeded perturbation, attacking the same ratchet from the
654
+ other side; measured gains reverse above 0.10, hence the widget cap.
655
 
656
  **Per component, not per latent** (fixed 2026-08-27). Video and audio are
657
+ two tensors in one NestedTensor and their statistics drift independently, so
658
  each carries its own anchor. Before this, `.std()` raised on the nested
659
+ object and every lever was dead -- announced once per hop as `pin
660
+ conditioning skipped`, which read as routine noise.
661
+
662
+ Returns `(latent, anchor)` where `anchor` is a list, one dict per component
663
+ -- the first pinned hop establishes what later hops are matched against.
664
+ Every lever defaults off, in which case the latent is returned untouched.
 
 
665
  """
666
  if not isinstance(lat, dict) or "samples" not in lat:
667
+ return lat, anchor
668
  x = lat["samples"]
669
  parts = _latent_parts(x)
670
  if parts is None:
671
  print(f"[{TAG}] pin conditioning skipped: unrecognised latent "
672
  f"({type(x).__name__})", flush=True)
673
+ return lat, anchor
674
+ mode = str(mode)
675
+ if mode == "on": # pre-2026-09-01 workflows
676
+ mode = "sigma"
677
  try:
678
+ cur = []
679
+ for t in parts:
680
+ sig = float(t.float().std())
681
+ cur.append({"sigma": sig, "ratio": _latents.band_ratio(t)})
682
  except Exception as e: # noqa: BLE001
683
  print(f"[{TAG}] pin conditioning skipped ({e!r})", flush=True)
684
+ return lat, anchor
685
+ if not all(c["sigma"] == c["sigma"] and c["sigma"] for c in cur):
686
+ return lat, anchor # zero or NaN in any stream
687
+ if anchor is None:
688
+ anchor = cur
689
+ if len(anchor) != len(cur):
690
  # Stream count changed mid-chain. Nothing sensible to match against.
691
  print(f"[{TAG}] pin conditioning skipped: latent has {len(cur)} "
692
+ f"component(s), anchor has {len(anchor)}", flush=True)
693
+ return lat, anchor
694
+
695
+ # Always report the drift, even with every lever off. This is the number
696
+ # that says whether a lever is needed and whether one worked, and it costs
697
+ # nothing to read -- the alternative is inferring it from the master after
698
+ # a decode, which is how the wrong statistic went unnoticed for a release.
699
+ for i, (c, a) in enumerate(zip(cur, anchor)):
700
+ if c["ratio"] is not None and a["ratio"]:
701
+ print(f"[{TAG}] pin drift[{i}]: sigma {c['sigma']:.4f} "
702
+ f"(x{c['sigma'] / a['sigma']:.4f} vs anchor) "
703
+ f"high-band fraction {c['ratio']:.4f} "
704
+ f"(x{c['ratio'] / a['ratio']:.4f})", flush=True)
705
+
706
+ if mode not in ("sigma", "band") and noise <= 0.0:
707
+ return lat, anchor
708
+
709
  out_parts, notes = [], []
710
+ for idx, (t, c, a) in enumerate(zip(parts, cur, anchor)):
711
  o = t
712
+ if mode == "sigma":
713
+ scale = a["sigma"] / c["sigma"]
714
  o = o * scale
715
+ notes.append(f"sigma[{idx}] x{scale:.4f}")
716
+ elif mode == "band":
717
+ o, k = _latents.match_band(o, a["ratio"])
718
+ if k is None:
719
+ # An audio component has no bands; leaving it alone is correct,
720
+ # not a fallback -- `sigma` on it would be a different lever
721
+ # applied silently under this one's name.
722
+ notes.append(f"band[{idx}] skipped (no spatial extent)")
723
+ else:
724
+ notes.append(f"band[{idx}] hi x{k:.4f} "
725
+ f"(fraction {c['ratio']:.4f} -> {a['ratio']:.4f})")
726
  if noise > 0.0:
727
  # Per component: `.shape` on the nested object reports only the
728
  # first component's shape, so one draw for the whole latent would
729
  # size its noise to the video and broadcast that onto the audio.
730
  g = torch.Generator(device="cpu").manual_seed((int(seed) + idx) & 0x7FFFFFFF)
731
  n = torch.randn(o.shape, generator=g, dtype=torch.float32)
732
+ o = o + n.to(dtype=o.dtype, device=o.device) * (float(noise) * a["sigma"])
733
  notes.append(f"noise[{idx}] {float(noise):.3f}")
734
  out_parts.append(o)
735
  if notes:
736
  print(f"[{TAG}] pin conditioning: " + ", ".join(notes), flush=True)
737
  new = dict(lat)
738
  new["samples"] = _rebuild_latent_samples(x, out_parts)
739
+ return new, anchor
740
 
741
 
742
  def _pin_mech_for(hop_index, overlap_n, prev_sampled):
 
1162
  "follows the overlap widget."
1163
  ),
1164
  }),
1165
+ "pin_renorm": (["off", "sigma", "band"], {
1166
  "default": "off",
1167
  "tooltip": (
1168
+ "Rescale each pinned latent back toward the first pinned "
1169
+ "hop's, to fight the texture ratchet -- measured at +4.2% "
1170
+ "mid-band per join, flat inside each hop. Both modes are "
1171
+ "scalar rescales, so neither moves structure or can blur "
1172
+ "detail. "
1173
+ "band: match the HIGH-BAND FRACTION, the statistic the "
1174
+ "ratchet actually moves. "
1175
+ "sigma: match total spread -- the original lever, kept "
1176
+ "for old workflows, and measurably the wrong statistic: "
1177
+ "total sigma FALLS across a chain whose picture is "
1178
+ "baking, so it corrects the wrong way. Saved as `on` "
1179
+ "before 0.5. "
1180
+ "off reproduces chain_00038 exactly. The log prints "
1181
+ "`pin drift` every hop either way, so you can read the "
1182
+ "ratchet without changing anything."
1183
  ),
1184
  }),
1185
  "pin_noise": ("FLOAT", {
 
1560
  pbar = comfy.utils.ProgressBar(n)
1561
 
1562
  hop_store = None
1563
+ # "on" is what pre-0.5 workflows saved for what is now "sigma".
1564
+ pin_renorm_mode = {"on": "sigma"}.get(str(pin_renorm), str(pin_renorm))
1565
+ if pin_renorm_mode not in ("sigma", "band"):
1566
+ pin_renorm_mode = "off"
1567
  pin_noise_v = max(0.0, min(0.10, float(pin_noise)))
1568
  audio_ctx = int(audio_pin_frames) if int(audio_pin_frames) > 0 else int(overlap_n)
1569
+ pin_anchor = None # the first pinned hop sets what 3+ match
1570
+ if pin_renorm_mode != "off" or pin_noise_v > 0.0:
1571
  print(f"[{TAG}] pin conditioning enabled: "
1572
+ f"renorm={pin_renorm_mode} "
1573
  f"noise={pin_noise_v:.3f}", flush=True)
1574
  tone_mode = str(tone_compensate)
1575
  tone_on = tone_mode != "off" and tone_mode in _tone.MODES
 
1803
  # has no sampler latent and falls back to AddGuide, which
1804
  # is a different render of the same inputs.
1805
  "pin_mech": pin_mech_pred,
1806
+ # Only from hop 2. Hop 1 has no pin -- `_pin_mech_for`
1807
+ # returns "none" for index 0 and the conditioning branch is
1808
+ # `elif i > 0` -- so its frames cannot depend on these
1809
+ # levers, and keying them in threw away a byte-identical
1810
+ # cached hop 1 every time one was flipped. That is a third
1811
+ # of the cost of every lever A/B, on the one hop nobody
1812
+ # needed to re-render.
1813
+ "pin_cond": ((pin_renorm_mode, round(pin_noise_v, 4), audio_ctx)
1814
+ if i > 0 else None),
1815
  })
1816
  # A locked shot reuses its last render even though its inputs
1817
  # changed -- that is the point of locking. The content key would
 
1862
  elif i > 0:
1863
  pin_latent = prev_sampled
1864
  if pin_latent is not None:
1865
+ pin_latent, pin_anchor = _condition_pin_latent(
1866
+ pin_latent, pin_anchor,
1867
+ mode=pin_renorm_mode, noise=pin_noise_v,
1868
  seed=(int(seed) + i))
1869
  cond, pin_mech_used = _pin_continue(
1870
  cond, latent, vae, audio_vae, overlap_n,
latents.py CHANGED
@@ -46,3 +46,100 @@ def from_dict(lat):
46
  if not isinstance(lat, dict) or "samples" not in lat:
47
  return None
48
  return parts(lat["samples"])
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  if not isinstance(lat, dict) or "samples" not in lat:
47
  return None
48
  return parts(lat["samples"])
49
+
50
+
51
+ def _gauss1d(sigma, device, dtype):
52
+ r = max(1, int(round(3.0 * float(sigma))))
53
+ x = torch.arange(-r, r + 1, dtype=torch.float32, device=device)
54
+ k = torch.exp(-(x * x) / (2.0 * float(sigma) ** 2))
55
+ return (k / k.sum()).to(dtype)
56
+
57
+
58
+ def band_split(t, sigma=2.0):
59
+ """Separable Gaussian low/high split over the last two dims. -> (lo, hi).
60
+
61
+ Returns None when the tensor has no spatial extent to speak of -- an audio
62
+ component is a different object and blurring across it means nothing.
63
+
64
+ Replicate padding, not reflect: a latent's spatial dims are small (a
65
+ 736x1280 frame is 46x80 here) and reflect needs the pad to be smaller than
66
+ the dimension, which stops being true for a large sigma on a small latent.
67
+ """
68
+ if t.dim() < 2 or t.shape[-1] < 8 or t.shape[-2] < 8:
69
+ return None
70
+ import torch.nn.functional as F # noqa: PLC0415
71
+ h, w = int(t.shape[-2]), int(t.shape[-1])
72
+ flat = t.reshape(-1, 1, h, w)
73
+ k = _gauss1d(sigma, t.device, t.dtype)
74
+ pad = k.numel() // 2
75
+ lo = F.conv2d(F.pad(flat, (pad, pad, 0, 0), mode="replicate"),
76
+ k.view(1, 1, 1, -1))
77
+ lo = F.conv2d(F.pad(lo, (0, 0, pad, pad), mode="replicate"),
78
+ k.view(1, 1, -1, 1))
79
+ lo = lo.reshape(t.shape)
80
+ return lo, t - lo
81
+
82
+
83
+ def band_ratio(t, sigma=2.0):
84
+ """High-band sigma as a fraction of total sigma. -> float, or None.
85
+
86
+ This is the statistic the texture ratchet actually moves. Measured across a
87
+ 3-hop chain it went 0.3643 -> 0.3673 -> 0.3702, monotone, while the total
88
+ sigma it is normalised by FELL 1.2% -- which is why matching total sigma
89
+ cannot see this and, worse, corrects the wrong way.
90
+ """
91
+ got = band_split(t, sigma)
92
+ if got is None:
93
+ return None
94
+ tot = float(t.float().std())
95
+ if not tot or tot != tot:
96
+ return None
97
+ return float(got[1].float().std()) / tot
98
+
99
+
100
+ def match_band(t, target_ratio, sigma=2.0, clamp=(0.5, 2.0)):
101
+ """Rescale t's high band so its high-band fraction becomes `target_ratio`.
102
+
103
+ -> (tensor, k), or (t, None) when the tensor has no bands to match.
104
+
105
+ Only the high half is scaled, so the low-frequency structure that carries
106
+ the scene is bit-identical and the correction is one scalar. It cannot blur,
107
+ sharpen unevenly, or invent detail; the worst it can do is get the gain
108
+ wrong, which is why `clamp` exists.
109
+
110
+ Note the fraction is against the tensor's own sigma, so restoring the ratio
111
+ does move total sigma a little. That is deliberate: the ratio is the drifting
112
+ statistic and sigma is the one that lies.
113
+ """
114
+ got = band_split(t, sigma)
115
+ if got is None or not target_ratio:
116
+ return t, None
117
+ lo, hi = got
118
+ cur_hi = float(hi.float().std())
119
+ if not cur_hi or cur_hi != cur_hi:
120
+ return t, None
121
+
122
+ # The naive `k = target * sigma / hi_sigma` is wrong, and quietly so:
123
+ # scaling the high band changes the sigma it is a fraction OF, so the
124
+ # target moves as you apply it. Measured, it undershot by 5% on a
125
+ # latent-shaped tensor -- a correction that silently does most, but not
126
+ # all, of its job is the worst kind to ship.
127
+ #
128
+ # First guess solves the fixed point assuming lo and hi are orthogonal:
129
+ # k*H / sqrt(L^2 + k^2*H^2) = r, so k = r*L / (H*sqrt(1 - r^2)) ...
130
+ r = min(float(target_ratio), 0.999)
131
+ lo_sd = float(lo.float().std())
132
+ k = (r * lo_sd) / (cur_hi * max(1e-6, (1.0 - r * r) ** 0.5))
133
+ # ... then refine against the statistic as actually measured, because a
134
+ # difference of Gaussians is not an exact orthogonal projection. Two or
135
+ # three passes converge, and a latent is small enough that this is free.
136
+ for _ in range(4):
137
+ k = min(max(k, clamp[0]), clamp[1])
138
+ got_r = band_ratio(lo + hi * k, sigma)
139
+ if not got_r:
140
+ break
141
+ if abs(got_r - r) <= 1e-4 * max(r, 1e-6):
142
+ break
143
+ k *= r / got_r
144
+ k = min(max(k, clamp[0]), clamp[1])
145
+ return lo + hi * k, k
tools/check_texture.py CHANGED
@@ -188,6 +188,71 @@ def main():
188
  ck("rebuild round-trips a plain tensor",
189
  torch.equal(lat.rebuild(t, [t]), t))
190
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
191
  print("\nhopcache.chains -- two renders stay two renders")
192
  mk = lambda hops: [(h, "k%d" % i, {"written": float(i)})
193
  for i, h in enumerate(hops)]
 
188
  ck("rebuild round-trips a plain tensor",
189
  torch.equal(lat.rebuild(t, [t]), t))
190
 
191
+ print()
192
+ print("latents.match_band -- the lever undoes a known tilt")
193
+ g = torch.Generator().manual_seed(3)
194
+ # Latent-SHAPED, not white noise. Plain randn has a high-band fraction of
195
+ # 0.966 -- pinned against its ceiling of 1.0, where lifting the high band
196
+ # barely moves the statistic and the clamp does the "correcting". Real
197
+ # latents measured 0.3643, so the fixture is built to land near there, and
198
+ # the first assertion holds it there.
199
+ w = torch.randn((2, 4, 46, 80), generator=g)
200
+ smooth = lat.band_split(w, 4.0)[0]
201
+ rough = torch.randn((2, 4, 46, 80), generator=g)
202
+ base = smooth / smooth.std() + rough / rough.std() * 0.30
203
+ r0 = lat.band_ratio(base)
204
+ ck("the fixture sits where real latents sit", 0.20 < r0 < 0.50,
205
+ "ratio %.4f (measured on a real chain: 0.3643)" % r0)
206
+
207
+ # Bake it: lift the high band the way a hop's pin drifts.
208
+ lo, hi = lat.band_split(base)
209
+ baked = lo + hi * 1.25
210
+ r1 = lat.band_ratio(baked)
211
+ ck("lifting the high band raises the fraction", r1 > r0 * 1.10,
212
+ "%.4f -> %.4f" % (r0, r1))
213
+ fixed, k = lat.match_band(baked, r0)
214
+ r2 = lat.band_ratio(fixed)
215
+ ck("match_band restores it to the anchor", abs(r2 / r0 - 1.0) < 0.005,
216
+ "%.4f vs %.4f (k=%.4f)" % (r2, r0, k))
217
+ ck("and scales the high band down to do it", k < 1.0, "k=%.4f" % k)
218
+
219
+ # The safety property -- but NOT "the low band is unchanged". Re-splitting
220
+ # the result does not hand back the same lo, because a difference of
221
+ # Gaussians is not an exact projection. What is true, and what makes this
222
+ # safe to run blind, is that the entire change lies along the high band:
223
+ # no structure is added, one scalar is moved.
224
+ d = (fixed - baked).reshape(-1)
225
+ hf = hi.reshape(-1)
226
+ cos = float((d * hf).sum() / (d.norm() * hf.norm()))
227
+ ck("the entire change lies along the high band", abs(cos) > 0.99,
228
+ "cos %.4f" % cos)
229
+ ck("an audio-shaped component is skipped, not sigma-matched",
230
+ lat.match_band(torch.randn(2, 1024), 0.4)[1] is None)
231
+ ck("a zero target is a no-op", lat.match_band(base, 0.0)[1] is None)
232
+
233
+ # The measured case reproduced: sigma FALLS while the fraction RISES, so
234
+ # the two levers disagree about which way to correct. This is the whole
235
+ # argument for `band` over `sigma` -- on the real chain, matching sigma
236
+ # would have scaled a pin UP whose high band was already too hot.
237
+ shrunk = baked * (0.99 * float(base.std()) / float(baked.std()))
238
+ ck("sigma and band disagree in DIRECTION on this signal",
239
+ float(shrunk.std()) < float(base.std())
240
+ and lat.band_ratio(shrunk) > r0 * 1.10,
241
+ "sigma x%.4f but fraction x%.4f"
242
+ % (float(shrunk.std()) / float(base.std()),
243
+ lat.band_ratio(shrunk) / r0))
244
+
245
+ # And the reason `sigma` cannot be rescued by tuning: the fraction is a
246
+ # RATIO, so it is invariant under any uniform rescale. Whatever scale a
247
+ # sigma match picks, it leaves this statistic exactly where it found it.
248
+ # That makes `sigma` a no-op on the texture ratchet by construction, not
249
+ # merely a weak correction -- measured end to end through the node, drift
250
+ # stayed at +12.74% under `sigma` and went to -0.04% under `band`.
251
+ for factor in (0.5, 0.9651, 2.0):
252
+ ck("a uniform x%.4g rescale does not move the fraction" % factor,
253
+ abs(lat.band_ratio(baked * factor) / r1 - 1.0) < 1e-4,
254
+ "%.6f vs %.6f" % (lat.band_ratio(baked * factor), r1))
255
+
256
  print("\nhopcache.chains -- two renders stay two renders")
257
  mk = lambda hops: [(h, "k%d" % i, {"written": float(i)})
258
  for i, h in enumerate(hops)]