US Patent 9788179 Detection and ranking of entities from mobile onscreen content

Systems and methods are provided for detecting and ranking entities identified in screen content displayed on a mobile device. For example, a method includes receiving an image captured from a mobile device display for a mobile application and determining a window that includes a chronological set of images, the images each representing a respective screen captured from a display of a mobile device and having an associated timestamp. The method also includes identifying entities appearing in images in a first portion of the window using text for images in a remaining portion of the window as context to disambiguate ambiguous entity references.