Free Online Courses for Software Developers - MrBool
× Please, log in to give us a feedback. Click here to login
×

You must be logged to download. Click here to login

×

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

×

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

Pairwise Testing: How to speed up the test execution

In this article we will discuss the details about Pairwise testing. It is one of the important testing methodologies used widely in testing industry and we will see how to speed up your test execution using Pairwise Testing.

Pairwise testing is a very powerful technique that helps a lot test analysts to speed up the test execution especially when they have to test one system or application with very entries and big test configurations.

The specification-based “Pairwise testing” technique can help a lot test analysts during test execution when test cases has multiple entries and consequently multiple combinations.

Normally, the test planning and execution in projects is ever short and analysts have to choose wisely the “correct” combinations in order to find as much defects and as soon as possible. To address this problem and guarantee the quality the release under testing, test analysts have to resort to specific techniques.

In this case, the most suitable technique would be “Pairwise Testing”. Basically, it consists in combining all possible entries of one specific system, using a smarter possible way in order to generate minimum test outputs as possible and maximizing the variation of the system entries. As this technique reduces the test cases combinations, it reduces the cost of the test execution. Pairwise Testing can be done manually using the “Orthogonal Array” concept or by some specific tools using the “Classification Tree” concept.

This technique has been used along the years ago in other areas as agricultural and medical experiments. In the 20s, some researchers started to use this technique in technology information. Along the years, the usage of this method has been brought reduced costs and quality to the projects.

Orthogonal Array technique:

This technique is done manually using pre-defined arrays in order to generate the test cases variations using the input system variables. Firstly, we have to choose the most suitable array. The number of the columns of this array will be the number of entries plus one. For example, if we have three entries, the selected array will have four columns.

Secondly, we take the entries possibilities and multiply the two largest numbers. The result of this operation will be the number of the lines of the array and meaning the number of possible test cases. After the array is selected, we have to do the correlations between the meaning of each line and column of the array. The number in the column means the order of the entries and the number in the line means the order the variation of the entries. After do the correlations, the test analyst can also remove one test combination if it is invalid. This must be done with very carefully.

Below, I will show an example of this technique. For example, we have to test one application, using the follow environments: Operational Systems (Mac, Linux, Windows 7); Browsers (Firefox, IE and Chrome) and in following Databases (Oracle, SQL Server and MySQL). As we have three entries, the suitable array must have four columns (3+1=4). In relation with lines, the array will have nine ones, because we have three variations of operational systems and databases as maximum entries variation (3*3 =9), meaning that nine test cases will be generated. The selected array is shown below (the value of the arrays is got from international standards).

Factors
Test 1 2 3 4
1 0 0 0 0
2 0 1 1 2
3 0 2 2 1
4 1 0 1 1
5 1 1 2 0
6 1 2 0 2
7 2 0 2 2
8 2 1 0 1
9 2 2 1 0

Table 1: Orthogonal Array

Doing the correlations, the result of combinations is shown below.

Factors
Test OS Browser Database
1 Mac Firefox Oracle
2 Mac IE SQL Server
3 Mac Chrome MySQL
4 Linux Firefox SQL Server
5 Linux IE MySQL
6 Linux Chrome Oracle
7 Windows7 Firefox MySQL
8 Windows7 IE Oracle
9 Windows7 Chrome SQL Server

Table 2: Test Cases Generated

Analyzing the results above we see that is impossible to test the combinations 2 and 5, because Internet Explorer is not supported with Mac and Linux operational systems. Only after that, quality analysts can remove these configurations from the test matrix above. Then, the final test combinations got from the technique is seven different combinations.

If test analysts do not use this technique, a lot of test combinations had to be done in order to guarantee the minimum quality of the product under testing. For this test, quality analysts will have to test twenty seven different environment combinations, because the supported environments are three versions of operational systems, browsers and databases (3*3*3=27). This number was reduced to only five guaranteeing the needed quality associated to this technique. Translating to the cost of the quality more than eighty per cent was reduced using this technique!!!

Classification Tree technique:

Instead raising the test combinations using orthogonal array manually, test analysts can use some tools that implement other technique that closely resemble the orthogonal array, the classification tree technique. This method is also used when test analysts have to test one system or feature with a lot of possible entries and have to test all combinations to find as many defects as soon as possible. To test all combinations would demand a lot of time and normally the time to test applications is very short.

Classification Tree technique has as base technical the theory that when pairs or triples of the variables of the system are combined and tested together, high percentage of the defects is quickly found (around 95% of all defects).

This theory is also used by other specification-based technique named “Equivalent Partitioning” and “Boundary Value Analysis”. Using these methods some results can be grouped into equivalent classes. Those classes have the same output characteristics when using a range of defined values. For example, to be approved one student has to get a score greater of equal to sixty. All test values between sixty and one hundred will exercise the equivalent class “approved” and values from zero to fifty-nine will exercise the class “reproved”. The theory associated with those is that where the most of the defects are found is exactly in the boundary values and to guarantee the coverage of the class, two or three values must be chosen. This quantity of the values is exactly the numbers that the Classification Tree technique implements, pairs or triples of input combinations.

There are a lot of available tools that implements this technique and test analysts can easily see graphically the optimum test scenarios when combining all pairs or triples of the entries of the system. I chose the best known named “CTE XL” tool (I will mention in the end of this article in References part how to get this tool).

To illustrate this technique, I will create a test example in which test analyst has to test one interface responsible to record user information. This interface has some numeric, alpha and date fields as Name, Phone, Address, City, Cep as pictures below show. First picture illustrate the test cases generation using pairs of entries and the second one illustrates test cases using triples of entries.

If triple entry combination will be used, more test cases will be generated. Test analysts have to find the balance between how much time is available to complete the testing and the risk associated in choosing one technique in comparison with other one.

Test Cases Generation using pair of entries

Figure 1: Test Cases Generation using pair of entries

Test Cases Generation using triple of entries

Figure 2: Test Cases Generation using triple of entries

References

Following are some references which can be followed to get more idea about pair wise testing.

  • Advanced Software Testing - Vol. 1, Rex Black.
  • Presentations on Star East conference:
    • Pairwise Testing Comes of Age, George B. Sheerwood, Ph.D.
    • The Bug Stops Here (Improving Test Design with a Novel Test Case Generator), Technical Presentation, Salon 6, May18.
  • Download the tool.


André Nascimento is the QA Lead in Hewlett-Packard in Porto Alegre, Brazil. He studied Control and Automation engineering at Catholic University of Minas Gerais and has after degree in software engineering and Telecommunication Sy...

What did you think of this post?
Services
[Close]
To have full access to this post (or download the associated files) you must have MrBool Credits.

  See the prices for this post in Mr.Bool Credits System below:

Individually – in this case the price for this post is US$ 0,00 (Buy it now)
in this case you will buy only this video by paying the full price with no discount.

Package of 10 credits - in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download few videos. In this plan you will receive a discount of 50% in each video. Subscribe for this package!

Package of 50 credits – in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download several videos. In this plan you will receive a discount of 83% in each video. Subscribe for this package!


> More info about MrBool Credits
[Close]
You must be logged to download.

Click here to login