US Patent 10977818 Machine learning based model localization system

A method for deriving an image sensor's 3D pose estimate from a 2D scene image input includes at least one Machine Learning algorithm trained a priori to generate a 3D depth map estimate from the 2D image input, which is used in conjunction with physical attributes of the source imaging device to make an accurate estimate of the imaging device 3D location and orientation relative to the 3D content of the imaged scene. The system may optionally employ additional Machine Learning algorithms to recognize objects within the scene to further infer contextual information about the scene, such as the image sensor pose estimate relative to the floor plane or the gravity vector. The resultant refined imaging device localization data can be applied to static (picture) or dynamic (video), 2D or 3D images, and is useful in many applications, most specifically for the purposes of improving the realism and accuracy of primarily static, but also dynamic Augmented Reality (AR) applications.