US Patent 12079731 Audiovisual source separation and localization using generative adversarial networks

A method (and structure and computer product) for an audiovisual source separation processing, including receiving video data including images of a plurality of sound sources, receiving an optical flow data of the video data, the optical flow data indicating motions of pixels between frames of the video data, and encoding the received video data into video localization data comprising information associating pixels in the frames of video data with different channels of sound.