International Business Machines Corporation
System testing infrastructure with hidden variable, hidden attribute, and hidden value detection

Abstract:

Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (S.sub.AV) is computed based on the binary execution results. An attribute is selected in response to a set of success rates corresponding to a set of attribute-value pairs that includes said attribute are all below a predetermined threshold. The set of testcases is executed using another set of test vectors using additional values for the selected attribute. For each execution of the set of testcases, for each attribute-value pair, a second non-binary success rate (S.sub.AV') is recorded. If the predetermined threshold is now satisfied, a user is notified of the additional values for the attribute that were detected.