Meta Platforms, Inc.
Techniques for dynamic throttling in batched bulk processing

Abstract:

Techniques for dynamic throttling in batched bulk processing are described. In one embodiment, an apparatus may comprise an execution management component operative to retrieve a plurality of batch-operation instructions from a batch-operation instruction store; initiate performance of the plurality of batch-operation instructions, wherein performance of the plurality of batch-operation instructions modifies data stored on a data storage shard of a data storage system; and throttle performance of the plurality of batch-operation instructions where one or more data operation performance signals indicate that the data storage shard is overloaded; and a storage monitoring component operative to monitor the one or more data operation performance signals for the data storage shard. Other embodiments are described and claimed.