US Patent 7920761 Multimodal identification and tracking of speakers in video

A computer program product includes machine readable instructions for providing enhanced video output by: receiving footage including likeness information in a plurality of modalities; demultiplexing the plurality of modalities to provide information for each modality; comparing information from at least two of the modalities for determining a correlation in the likeness information; using the correlation, obtaining semantic information for association with the likeness; and combining the semantic information with the likeness information for providing the enhanced video output. A system for implementing the computer program product includes resources for receiving the footage.