US Patent 10515313 Predictive model evaluation and training based on utility

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a plurality of different types of predictive models using training data, wherein each of the predictive models implements a different machine learning technique. One or more weights are obtained wherein each weight is associated with an answer category in the plurality of examples. A weighted accuracy is calculated for each of the predictive models using the one or more weights.