Visa Inc.
System, Method, and Computer Program Product for Determining a Reason for a Deep Learning Model Output

Abstract:

A system, method, and product for determining a reason for a deep learning model output that obtain training data associated with training samples and first labels for the training samples; train a first model using the training samples and the first labels, training the first model generating predictions for the training samples; train a second model using the training samples and the predictions as second labels for the training samples; extract one or more weights of the trained second model; process, using the first model, input data including features associated with at least one sample, to generate output data, the output data including at least one prediction for the at least one sample; and apply the one or more extracted weights to the features to determine one or more contributions of one or more features of the features to the at least one prediction for the at least one sample.