LendingClub Corporation
DISTRIBUTED JOB FRAMEWORK AND TASK QUEUE

Abstract:

A queue-based task management system is provided. Unlike conventional queue-based task management systems, the system described herein does not handle all tasks in the conventional manner. Rather, tasks can be associated with one of several modes, including: a queue-mode (always storing the task in the queue) and an immediate-mode (record the task, but execute it immediately if possible). The two modes may be controlled programmatically to optimize utilization of system resources. Immediate-mode is implemented by monitoring available task system resources, and executing the task immediately in-process if there are resources available; otherwise the task is delegated to the queue. This in-process execution allows the sharing of parent task resources.