US Patent 10019831 Integrating real world conditions into virtual imagery

Systems and methods for incorporating real world conditions into a three-dimensional (3D) graphics object are described herein. In some embodiments, images of a physical location of a user of a three-dimensional (3D) display system may be received from at least one camera and a data imagery map of the physical location may be determined based at least in part on the received images. The data imagery map may capture real world conditions associated with the physical location of the user. Instructions to render a 3D graphics object may be generated and the data imagery map may be incorporated into a virtual 3D scene comprising the 3D graphics object, thereby incorporating the real world conditions into virtual world imagery. In some embodiments, the data imagery may include a light map, a sparse light field, and/or a depth map of the physical location.