US Patent 11518406 Test failure detection using a governing agent data set

Systems and methods for detecting out-of-bounds behavior of an agent in a simulation including multiple agents and an ego vehicle may include: providing a scenario governing behavior of an agent in the simulation; operating the simulation in accordance with the scenario across multiple occurrences; collecting data regarding behavior of the agent in the scenario for each occurrence; comparing collected data regarding behavior of the agent in the scenario with a governing data set for the scenario; and reporting an out-of-bounds condition to a system user when the results of the comparison indicate that the behavior of the agent in the scenario deviates from the agent behavior in the governing data set scenario by more than a predetermined amount.