Dropbox, Inc.
Job-processing systems and methods with inferred dependencies between jobs

Abstract:

An analytics system that executes processing jobs infers dependencies between jobs to be executed based on identification of dependencies between a "sink" job and a source data object on which the sink job depends. Given a job definition for the sink job that identifies a source data object, the system can identify a "source" job that produces the source data object and can infer a dependency of the sink job on the source job. The system can schedule executions of the source and sink jobs such that the source job completes (or completes generation of the source data object) before the sink job is launched.