International Business Machines Corporation
OPTIMIZING A MACHINE LEARNING SYSTEM

Abstract:

A method optimizes machine learning systems. A computing device accesses a committee of classifiers that have been trained using an initial labeled instance of data from an annotator. The initial labeled instance of data includes annotator-ranked attributes of the data, initial values of the attributes, and an initial prediction label that describes an initial predicted state based on the values. The computing system compares the attributes ranking from the annotator to attributes rankings that are generated by and used by each of the machine learning systems when evaluating one or more instances of unlabeled data that include the attributes, and weights the machine learning systems according to how closely each of the attributes rankings generated by and used by each of the machine learning systems match the attributes ranking from the annotator. The machine learning systems are then optimized based on this matching.