US Patent 7577709 Reliability measure for a classifier

In one aspect, a data item is input into a scoring classifier such that the scoring classifier indicates that the data item belongs to a first class. A determination is made as to the amount of retraining of the scoring classifier, based on the data item, that is required to cause the scoring classifier to indicate that the data item belongs to a second class. A reliability measure is determined based on the required amount of retraining and a class of the data item is determined based, at least in part, on the reliability measure.