US Patent 11531347 System and method to virtually teach a semi-autonomous device

A virtual teaching system includes a semi-autonomous device to perform a task such as cleaning a target environment. The semi-autonomous device includes one or more sensors configured to record environmental data from the target environment that can be used to construct a virtual environment. The semi-autonomous device is operably coupled to an analysis system. The analysis system includes a processor to perform multiple functions, such as constructing the virtual environment from the recorded environmental data and supporting operation of a user interface. The user interface can be operably coupled to the processor, allowing a human operator to teach a virtual device in the virtual environment to perform an action sequence. Once the virtual device has been taught an action sequence in the virtual environment, the analysis system can transfer the recorded action sequence to the semi-autonomous device for use in the target environment.