US Patent 9335764 Virtual and augmented reality cockpit and operational control systems

Architecture for a multimodal, multiplatform switching, unmanned vehicle (UV) swarm system which can execute missions in diverse environments. The architecture includes onboard and ground processors to handle and integrate multiple sensor inputs generating a unique UV pilot experience for a remote drone pilot (RDP) via a virtual augmented reality cockpit (VARC). The RDP is monitored by an operational control system and an experienced control pilot. A ground processor handles real-time localization, forwarding of commands, generation and delivery of augmented content to users, along with safety features and overrides. The UVs onboard processors and autopilot execute the commands and provide a redundant source of safety features and override in the case of loss of signal. The UVs perform customizable missions, with adjustable rules for differing skill levels. RDPs experience real-time virtual piloting of the UV with augmented interactive and actionable visual and audio content delivered to them via VARC systems.