US Patent 10031526 Vision-based driving scenario generator for autonomous driving simulation

Described is a system (and method) for generating a driving scenario for an autonomous driving simulator. The system may use a camera mounted to a vehicle as a cost effective approach to obtain real-life driving scenario data. The system may then analyze the two-dimensional image data to create a three-dimensional driving simulation. The analysis may include detecting objects (e.g. vehicles, pedestrians, etc.) within the two-dimensional image data and determining movements of the object based on a position, trajectory, and velocity of the object. The determined information of the object may then be projected onto a map that may be used for generating the three-dimensional driving simulation. The use of cost-effective cameras provides the ability to obtain vast amounts of driving image data that may be used to provide an extensive coverage of the potential types of driving scenarios an autonomous vehicle may encounter.