Patent attributes
A system and method for performing regression testing on source code provides a data sample to one or more candidate instances executing a candidate version of source code for a code base and provides the data sample to plurality of baseline instances executing a baseline version of the source code. The baseline version of the source code and the candidate version of the source code differ. The regression testing system and method obtains candidate outputs from the one or more candidate instances and obtains baseline outputs from the plurality of baseline instances based on the provided data sample. One or more faults in the candidate version of the source code can be determined by comparing the candidate outputs the baseline outputs. A filter for excluding one or more data fields from the comparison can be generated based on a comparison between baseline outputs from one of the plurality of baseline instances and baseline outputs from another of the plurality of baseline instances. Faults can also be determined based on satisfaction of a sample count, a relative threshold, an absolute threshold, or a median absolute deviation comparison.
