Microsoft Corporation
Automatically inferring data relationships of datasets

Abstract:

Described herein is a system and method for inferring data relationships of a plurality of datasets. Data contents (and optionally metadata) of the plurality of datasets are scanned to extract features of each of the datasets. Features can be related to a structure of data, a profile of data within the dataset, and/or metadata of the dataset. Each feature has an associated weight. The datasets can be clustered into clusters based on at least some of the weighted features (e.g., based on a sim-hash or min-hash of the dataset). A precise similarity metric is computed between datasets in each cluster based on their weighted features. Datasets with precise similarity metrics above a threshold quantity are inferred to be being likely related. Information is provided regarding the inferred likely related datasets.