Palantir Technologies Inc.
SYSTEMS AND METHODS FOR CONSISTENT BACKUP OF DISTRIBUTED, TRANSACTIONAL DATABASES

Abstract:

A distributed, transactional database uses timestamps, such as logical clock values, for entry versioning and transaction management in the database. To write to the database, a service requests a timestamp to be inserted into the database with a new version of data. During a backup procedure, a cleanup process is paused, issuing new timestamps is paused, and a backup timestamp is generated, which results in an effective backup copy. Such steps approximate turning off all of the database servers and backing up the servers due to the external devices and services being unable to write new data for a period of time and having the cleanup turned off for a period of time. A snapshot of the database can have some inconsistent entries. During a restore of a backup, a snapshot of the database is loaded and any entries older than the backup timestamp are deleted, which ensures that a consistent restore has occurred. Uploads of backup copies can be expedited by only uploading incremental changes and keeping track of data that is associated with multiple backup copy versions.