Adobe Inc.
SCALABLE QUERYING FOR DISTRIBUTED SYSTEMS

Abstract:

A flexible, scalable, and fast query system for a distributed execution environment is provided. An example method includes receiving a plurality of processor summaries, each including a list of job digests, each job digest including a job identifier for a job running on a job processor and a payload portion. The method can also include, for each job digest, determining a location in a probabilistic payload data structure for the job digest and storing the payload portion and a fingerprint portion generated from a portion of a hash of the job identifier in an entry at the location as a job summary and responding to queries using the probabilistic payload data structure. Responding to a particular query may include identifying job properties corresponding to parameters of the particular query, updating job property statistics, and using the job property statistics to change the job properties represented in the payload portion.