US Patent 11361519 Interactable augmented and virtual reality experience

A client device can display various augmented reality and virtual reality images to a user. For example, the client device captures images of an external environment and identifies objects in the environment. If an object satisfies a virtual environment criterion, the client device displays a window to a virtual environment at the location of the object in the environment. In another example, the client device includes a light assembly that can generate light. If the user turns the light assembly on, virtual objects displayed in the environment will gather to a region highlighted by the light from the light assembly. In another example, the client device estimates depth information of an object in the environment. The client device then displays a virtual object in the environment based on the depth information and may also display the virtual object interacting with an object in the external environment.