Systems and methods for robust biometric applications using a detailed eye shape model are described. In one aspect, after receiving an eye image of an eye (e.g., from an eye-tracking camera on an augmented reality display device), an eye shape (e.g., upper or lower eyelids, an iris, or a pupil) of the eye in the eye image is calculated using cascaded shape regression methods. Eye features related to the estimated eye shape can then be determined and used in biometric applications, such as gaze estimation or biometric identification or authentication.