International Business Machines Corporation
METHOD AND SYSTEM FOR PERFORMING DATA PRE-PROCESSING OPERATIONS DURING DATA PREPARATION OF A MACHINE LEARNING LIFECYCLE

Abstract:

In accordance with an embodiment of the invention, a method is provided for performing data pre-processing operations during data preparation of a machine learning lifecycle. The method includes defining one or more data pre-processing functions for applying to data stored in a dataset, executing one or more learn functions for learning the data, and executing one or more transform functions for transforming the data. Each of the one or more learn functions generates a first Structured Query Language (SQL) statement representing a definition of corresponding learn function for corresponding defined data pre-processing function. Each of the one or more transform functions generates a second SQL statement representing a definition of corresponding transform function for corresponding defined data pre-processing function. The dataset is stored in a database.