Twilio Inc.
HIERARCHICAL SCHEDULER

Abstract:

Methods, systems, and computer programs are directed to the implementation of configurable hierarchical schedulers with multiple levels, where each level may use one of several types of queueing mechanisms. A configurable, hierarchical scheduler is designed to handle large scale processing of requests (e.g., transmitting outgoing messages). The hierarchical scheduler distributes the loads to different queues handling different types of messages (e.g., by user ID, by Internet Address (IP), by schedule). The different layers of the hierarchical scheduler are configurable to queue and schedule traffic based on many factors, such as IP address, handling reputation, available downstream bandwidth, fairness, concurrency rates to handle multiple constraints, scheduling per client, time of delivery constrains, rate limits per user, domain scheduling per user, concurrency throttling per outbound channel, and sharing global rate limits across service processors.