VMware, Inc.
Using data rebuilding to support large segments

Abstract:

Techniques for supporting large segments when issuing writes to an erasure coded storage object in a distributed storage system are provided. In one set of embodiments, a node of the system can receive a write request for updating a logical data block of the storage object, write data/metadata for the block to a record in a data log of a metadata object of the storage object (where the metadata object is stored on a performance storage tier), and determine whether the data log has accumulated a threshold number of records. If so, the node can further allocate an in-memory bank, place the data from the data log records into free slots of the bank, allocate a segment in a capacity object of the storage object for holding contents of the bank (where the capacity object is stored on a capacity storage tier), and write the bank contents via a full stripe write to the allocated segment.