International Business Machines Corporation
Method to measure similarity of datasets for given AI task

Abstract:

A computer-implemented method comprises: inputting into an autoencoder sets of input samples, each of the sets of input samples comprising: a reference input sample of a reference dataset and one or more target input samples of one or more target datasets, the autoencoder being trained using the reference dataset. The autoencoder generates a respective set of outputs for each set of the input samples to thereby form one or more respective sets of outputs, each of the one or more sets of outputs comprising the reference output and the one or more target outputs for a respective set of input samples; and determining the similarity of each of the one or more target datasets to the reference dataset by comparing each of the one or more target outputs to respective target input samples of each of the sets of input samples.