Sandpies commited on
Commit
efd6a3e
·
1 Parent(s): 6c5d3ea

media: decode audio with PyAV, because torchaudio.load no longer decodes

Browse files

The soundtrack never played, and the console said why:

could not read audio 'Track 2 master test.mp3': ImportError('TorchCodec
is required for load_with_torchcodec. Please install torchcodec ...')

torchaudio 2.9 removed its own decoding backends and left `load` a thin
wrapper over torchcodec, which is not installed here. So `load_audio` was
raising for every file, wav and mp3 alike.

This was never soundtrack-specific -- `voice_file` goes through the same
function, so the voice reference has been failing on this venv too, and the
only symptom was a note in the console and a chain that quietly generated its
own voice. The soundtrack is what made it visible.

Fixed by decoding with PyAV, which is already a hard ComfyUI dependency and
is what core's own Load Audio uses, so a file picked in the panel now takes
exactly the same path as one arriving down a wire. Not by installing
torchcodec: that is another unsigned binary wheel on a machine where Smart
App Control just blocked PyAV's, and it would fix one venv rather than the
node.

torchaudio stays for `functional.resample` in music.py. It is only the
decoding half of that library that is gone.

Verified on the user's own 173 s 48 kHz mp3: decodes to [1, 2, 8327688],
and music.apply loops it down to a 9.4 s chain with the bed measuring 3x
louder in the speech gaps than under the speech.

Files changed (1) hide show
  1. media.py +42 -5
media.py CHANGED
@@ -207,18 +207,55 @@ def load_audio(name):
207
 
208
  Shape is `[batch, channels, samples]`, which is what every AUDIO consumer
209
  in the graph expects.
 
 
 
 
 
 
 
 
 
 
 
210
  """
211
  path = resolve(name, kinds={"audio"})
212
  if path is None:
213
  return None
214
  try:
 
215
  import torch # noqa: PLC0415
216
- import torchaudio # noqa: PLC0415
217
 
218
- wav, sr = torchaudio.load(path)
219
- if wav.dim() == 1:
220
- wav = wav.unsqueeze(0)
221
- return {"waveform": wav.unsqueeze(0).float(), "sample_rate": int(sr)}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
222
  except Exception as exc:
223
  print(f"[{TAG}] could not read audio {name!r}: {exc!r}", flush=True)
224
  return None
 
207
 
208
  Shape is `[batch, channels, samples]`, which is what every AUDIO consumer
209
  in the graph expects.
210
+
211
+ Decoded with PyAV rather than `torchaudio.load`. torchaudio 2.9 removed its
212
+ own decoding backends and left `load` a thin wrapper over `torchcodec`, so
213
+ on an install without that package -- including this one -- it raises
214
+ ImportError for every file, wav and mp3 alike, and the only symptom is a
215
+ reference that silently does not arrive. PyAV is already a hard ComfyUI
216
+ dependency and is what core's own Load Audio decodes with, so a file picked
217
+ in the panel now takes exactly the same path as one arriving down a wire.
218
+
219
+ `torchaudio` is still used for `functional.resample` in music.py; it is only
220
+ the *decoding* half of that library that is gone.
221
  """
222
  path = resolve(name, kinds={"audio"})
223
  if path is None:
224
  return None
225
  try:
226
+ import av # noqa: PLC0415
227
  import torch # noqa: PLC0415
 
228
 
229
+ with av.open(path) as container:
230
+ if not container.streams.audio:
231
+ print(f"[{TAG}] {name!r} has no audio stream", flush=True)
232
+ return None
233
+ stream = container.streams.audio[0]
234
+ sr = int(stream.codec_context.sample_rate)
235
+ channels = int(stream.channels)
236
+ chunks = []
237
+ for frame in container.decode(streams=stream.index):
238
+ buf = torch.from_numpy(frame.to_ndarray())
239
+ # Planar formats decode to [channels, samples]; packed ones to
240
+ # [1, samples*channels] interleaved. Same reshape ComfyUI uses.
241
+ if buf.shape[0] != channels:
242
+ buf = buf.view(-1, channels).t()
243
+ chunks.append(buf)
244
+ if not chunks:
245
+ print(f"[{TAG}] {name!r} decoded to no audio frames", flush=True)
246
+ return None
247
+ wav = torch.cat(chunks, dim=1)
248
+ # Integer PCM is scaled by its own full range, not normalised by peak:
249
+ # a quiet take must stay quiet, and dividing by max would silently
250
+ # apply a wildly different gain per file.
251
+ if not wav.dtype.is_floating_point:
252
+ if wav.dtype == torch.int16:
253
+ wav = wav.float() / (2 ** 15)
254
+ elif wav.dtype == torch.int32:
255
+ wav = wav.float() / (2 ** 31)
256
+ else:
257
+ wav = wav.float()
258
+ return {"waveform": wav.float().unsqueeze(0), "sample_rate": sr}
259
  except Exception as exc:
260
  print(f"[{TAG}] could not read audio {name!r}: {exc!r}", flush=True)
261
  return None