Shopify Inc.
SYSTEM AND METHOD OF PROCESSING LATE ARRIVING AND OUT OF ORDER DATA

Abstract:

Systems and methods for processing out of order data incrementally are provided. A database is maintained containing rows of data, each row of data having a timestamp and pertaining to a transaction, for example in the e-commerce platform. New data for new rows of data is received. At least some of the data is out of order. Each new row of data is processed in the same manner irrespective of whether the row is out of order or in order using a computation graph including at least one execution node configured to perform out-of-order incremental processing. A processing result is output based on the processing, wherein the result is up to date based on data that has been received.