NICE Ltd.
STAGING TABLES FOR DATA INGESTION OF MICRO-BATCHED DATA RECORDS

Abstract:

A system is provided to provide near real-time updates to data tables for a data warehouse using a micro-batch manager. The system includes a processor and a computer readable medium operably coupled thereto, to perform table updating operations which include accessing a first staging data table comprising a plurality of records that was generated by the micro-batch manager for a first destination table of the data tables for the data warehouse, reading a merge instruction column of the first staging data table for the plurality of records, determining a merge operation for each of the plurality of records from reading the merge instruction column, merging the plurality of records from the first staging data table to the first destination table using the merge operation for each of the plurality of records, and deleting the plurality of records from the first staging data table after merging.