Microsoft Corporation
Using lineage to infer data quality issues

Abstract:

Identifying data quality along a data flow. A method includes identifying quality metadata for two or more datasets. The quality metadata defines one or more of quality of a data source, accuracy of a dataset, completeness of a dataset, freshness of a dataset, or relevance of a dataset. At least some of the metadata is based on results of operations along a data flow. Based on the metadata, the method includes creating one or more quality indexes for the datasets. The one or more quality indexes include a characterization of quality of two or more datasets.