US Patent 10445936 Audio responsive augmented reality

Systems, devices, media, and methods are presented for generating responsive augmented reality elements in a graphical user interface. The systems and methods receive a video stream having one or more frames containing image data and audio data. The systems and methods identify a set of coordinates within a portion of the one or more frames and identify one or more audio characteristics within the audio data of the video stream. In response to the one or more audio characteristics, the systems and methods generate one or more graphical interface elements and detect a change in the audio data within the video stream. The systems and methods modify the one or more graphical interface elements in a second portion of the video stream in response to the change in the audio data.