US Patent 11651037 Efficient cross-modal retrieval via deep binary hashing and quantization

The present disclosure relates to a new method for cross-modal retrieval via deep binary hashing and quantization. In a training phase, the system simultaneously learns to generate feature vectors, binary codes, and quantization codes for data across two or more modalities that preserves the semantic similarity correlations in the original data. In a prediction phase, the system retrieve a data item in a database that is semantically similar to a query item of a different modality. To identify the database item closest in semantic meaning to the query item, the system first narrows the database search space based on binary hash code distances between each of the database items and the query item. The system then measures the quantization distances between the query items and the database items in the smaller search space. The system identifies database item have the closest quantization distance to the query item as the closest semantic match to the query item.