VMware, Inc.
SCALABLE AND DYNAMIC DATA COLLECTION AND PROCESSING

Abstract:

Examples described herein include systems and methods performing scalable and dynamic data processing and extraction. A first example method relates to processing events from a source. The method can include detecting an event generated by the source and predicting a probability of that event being part of a span including multiple events. The method can include waiting for the additional multiple events to occur within the predicted timeframe and, if occurring, packaging the events together for handling by a single dynamic function. Otherwise, the events can each be handled by separate dynamic functions. A second example method relates to performing dynamic data extraction from a source. The method can include waking up a function based on a regular poll interval, determining a probability of a data change at the source based on historical data extractions, and invoking an extraction function based on the probability of the data change.