US Patent 11238340 Predictive eyetracking using recurrent neural networks

A system predicts future positions or vergence depth of the user's eyes and generates gaze contingent content, such as for a head-mounted display (HMD), based on the predicted positions or vergence depth. The system includes an eye tracking controller that creates eye tracking information defining positions of a first eye and a second eye of a user over time. The eye tracking information is input to a neural network model that outputs the predicted positions or vergence depth. The predicted positions or vergence depth is then used to render the gaze contingent content, or to change other configurations of the HMD. Latency between the detection of user eye movement and the output of corresponding content is reduced to provide a more immersive real-time user experience.