This article is hopefully useful for those who are looking for a method to perform all-pair matching testing, in order to reduce the number of test cases, save time and effort.
Many people believe that testing software as much as possible, believe that it is important to include all possible states and variables, ensuring that all errors are found. In reality, there is not enough time, testers or efforts to test every combination of all variables. Not all defects will be found, making quality assurance risk management.
So how to know that our product is ready for release within reasonable time and cost? That is, how to manage the risk of not checking everything? One solution that we are going to find out is a ‘All-Pairs Testing’ technique called ‘Pairwise Testing’ also known as ‘Combinatorial Testing’.
Definition: What is All-pairs Testing?
All-pairs also known as pairwise testing is a test design technique for testing the software using combinatorial methods. It's a method to test all possible discrete combinations of the relevant parameters, the least test method so that the best quality.
In fact, smart testing is very necessary and important. Most of the errors are caused by the combination of 2 factors (parameters). So we need to test all pairs which produces a test set containing all the test value pairs of the variables. That techniques should be effective for high defect yield rate and maximum test coverage. And pairwise testing will help you to significantly reduce the number of test cases and still be effective in detecting errors (50-90%).
All-pairs Testing Example
Suppose there is a software function to be tested where there are already 10 input fields and 10 possible values for each input field. Then there will be 10^10 inputs that can be tested. In this case, a comprehensive check is not possible.
To better understand the concept of checking all pairs, let's consider Book Ordering Application example:
The book ordering application allows for buying books and selling books. It supports trading in some cities like Hanoi, Hochiminh City, and Da Nang. It should allow the trade of following books: Action, Classics, Novel, ... Two types of booking: Ecommerce and In Store. And orders can be done only during trading hours.
Step 1: List down all the variables involved.
1. Order category
+ Buying book
+ Selling book
2. City
+ Hanoi
+ Hochiminh City
+ Da Nang
3. Book types
+ Action
+ Classics
+ Novel
+ … more than 50 types
4. Booking type
+ Ecommerce
+ In Store
5. Booking time
+ Working hours
+ Non-working hours
If we want to test all valid combinations: 2 x 3 x 50 x 2 x 2 = 1200 valid test cases. Total number of test cases including possible negative cases > 1200 cases. Thus, it will take a long time to fully test.
Step 2: Simplify
Reduce Book types to two:
- Valid type (in list)
- Invalid type (not in list)
Now calculate the number of combinations again: 2 X 3 X 2 X 2 X 2 = 48 valid test cases
Step 3: Order the variables
The variable with the most value ranges is sorted first, and the variable with the least number of value ranges is left last.
Step 4: Create a test suite
Fill in the corresponding range of values in the table by column. The values of City (highest number of values) should be written by the number of times of the next highest variable ( two time for Order category).
Continue to fill in the value ranges for the Order category column (2 values).
Repeat the same for column Book types, Booking types and Booking time.
In this step, we see that, there’s no Buying book with Invalid type and there’s no Selling book with Valid type. Check to make sure that all combinations are covered. Rearrange as follows:
We will repeat the same steps for Book types and Booking type.
Each value in Book types has both the values of Booking type. But each value in Order category does not have two values in Booking type. We need to rearrange the last set of values in the Booking type as follows:
Continue with Booking time.
Test results
+ Fully combined results in >1200 cases.
+ Normal technical results of software testing 48 test cases.
+ All-pairs Technical results of Software testing in only 6 test cases.
Conclusion
Paired testing technique allows to ensure proper testing, can greatly reduce the number of combinations mentioned and does not need to test every possible combination but still ensures efficiency in terms of error detection.
Testing all pairs can reduce the number of tests running by >=75%. Along with saving time, test designers will be able to spend more time developing more complex test cases.
After all, this technique should be considered a priority in test design, shouldn't it?
SHIFT ASIA, we provide a variety of high-quality testing solutions to fit our clients' needs. For more details about our testing solutions, please click the following banner.
