US Patent 11507619 Display apparatus with intelligent user interface

A display apparatus includes user input circuitry for receiving user commands and a display for displaying video content and a user interface. The video content includes metadata. The display apparatus includes a processor in communication with the user input circuitry and the display; and non-transitory computer readable media in communication with the processor that stores instruction code. The instruction code, when executed by the processor, causes the processor to receive, from the user input circuitry, a query regarding an image of the video content currently displayed on the display; and determine one or more objects of the image associated with the query based on the metadata. The processor updates the user interface to depict one or more controls, each control being associated with one of the determined one or more objects; and receives a selection of one of the controls. The processor then updates the user interface to depict information related to the selection.