VMware, Inc.
Content-based post-process data deduplication

Abstract:

Techniques for implementing content-based post-process data deduplication are provided. In one set of embodiments, a computer system can receive a write request comprising write data to be persisted to a storage system and can sample a portion of the write data. The computer system can further execute one or more analyses on the sampled portion in order to determine whether the write data is a good deduplication candidate that is likely to contain redundancies which can be eliminated via data deduplication. If the one or more analyses indicate that the write data is a good deduplication candidate, the computer system can cause the write data to be persisted to a staging storage component of the storage system. Otherwise, the computer system can cause the write data to be persisted to a primary storage component of the storage system that is separate from the staging storage component.