Oracle Corporation
System and method for providing a hybrid set-based extract, load, and transformation of data

Abstract:

In accordance with an embodiment, described herein is a system and method for loading and transforming data to a cloud database, using a hybrid set-based extract, load, transform approach. During the loading and transforming of large amounts of data, from one or more data sources, a data synchronization application sends small batches of data, in parallel streams, to the database, which operates as a transformation engine to load the data. Each stream can be associated with a separate staging table in the database. For each staging table, the subset of data therein is transformed and loaded, before the next subset is processed. A transform and merge process operating at the database can then be used to transform and merge the data, from each of the staging tables, to the target table.