The procedure for minimizing a set of tests aims to select the minimum (with respect to the number of tests) subset of tests T needed to cover each program element that depends on the changes. To verify the correctness of the program, only tests from the minimal subset are used.
The rationale for applying minimization methods is as follows:
- The correlation between the efficiency of error detection and code coverage is higher than that between the efficiency of error detection and the size of a test set. Ineffective testing, such as long running tests that do not cover most of the code, can lead to an erroneous conclusion about the correctness of the program.
Take a look at list software testing companies to familiarize yourself with experienced qa specialists and choose the one to get testing work done for you.
- Regardless of the way the original test suite is generated, its minimal subsets have the advantage due to their size and efficiency, since they consist of fewer tests that have the same or a little bit lower ability to detect errors.
- Generally speaking, a shortened set of tests selected using the minimization method can detect errors that are not detected by a shortened test set of the same size chosen randomly or in some other way. It is quite natural that the minimization method is more efficient than random methods. However, among all the deterministic methods, the minimization leads to creating the least effective sets of tests, although the smallest ones. In particular, safe methods are more effective than minimization methods, although they are much more expensive.
Minimization of test set involves additional expenses since it requires some level of evaluation. If the cost of this analysis is greater than the cost of executing a certain threshold number of tests, there is a cheaper random method that provides the same error detection efficiency.
Although minimal test suites can provide structural coverage for the changed code, they are often not safe because it is obvious that some tests potentially capable of detecting errors can be rejected from the selection process. A set of functional tests is not usually redundant in the sense that no two tests cover the same functional requirements. If the tests were originally created using the structural coverage criterion, the minimization is fruitful, but when we are dealing with functional tests, it is preferable not to discard tests potentially capable of detecting errors. In the existing testing practice, engineers prefer not to minimize the test suite.