International Business Machines Corporation
Method and System for Crowdsourced Proactive Testing of Log Classification Models

Abstract:

A system configured to proactively test a log classification model using crowdsourcing, the system comprising memory for storing instructions, and a processor configured to execute the instructions to receive the log classification model as input; provide an explanation for predictions made by the log classification model to a crowd; receive a test sample from a first crowd worker, the test sample is intended to generate an error for the log classification model; generate a prediction using the log classification model based on the test sample as input; receiving validation data corresponding to the prediction of the log classification model; receive categorization data of the error corresponding to the test sample; and improve the log classification model based on the test sample.