US Patent 9569701 Interactive text recognition by a head-mounted device

A user is assisted in a real-world environment. An assistance engine receives at least one context from the user. The assistance engine also receives a video stream of the real-world environment. The assistance engine performs an optical character recognition process on the video stream based upon the at least one context. The assistance engine generates a response for the user. A microphone on a head-mounted device receives the context from the user. A camera on the head-mounted device captures the video stream of the real-world environment. A speaker on the head-mounted device communicates the response to the user. The user may move in the real-world environment based upon the response to improve the optical character recognition process.