Intuit Inc.
BATCH TO STREAM PROCESSING IN A FEATURE MANAGEMENT PLATFORM

Abstract:

Certain aspects of the present disclosure provide techniques for a "hand-off" operation of a feature management platform. A feature management platform can receive a request to generate feature data based on batch and streaming data. To generate such feature data, a "hand-off" occurs between a batch processing job to a stream processing job. The feature management platform can initiate the batch processing job to generate a first set of feature data. Once all of the feature data is generated by the batch processing job, the feature data is saved in an offline database. The feature data with the maximum timestamp is saved in an online database, and the maximum timestamp is saved in a persistent database. With the maximum timestamp, the feature management platform begins the stream processing job. Once feature data is generated by the stream processing job, the feature data is stored in an offline or online database.