Testing is one of the key phases in software development life cycle (SDLC). Experience shows that around 25 - 30% of the total SDLC effort is goes towards the testing phase. There are certain tools and techniques developed to optimize various phases of testing. In this article, we will focus on optimization of test suite using orthogonal array.

Orthogonal Array (OA):
Orthogonal arrays are two dimensional arrays of numbers which possess the interesting quality that by choosing any two columns in the array you receive an even distribution of all the pair-wise combinations of values in the array. However, this is applicable only for orthogonal arrays of strength 2. Orthogonal arrays with different strengths would require a different number of columns to select the tests and would not necessarily be only concerned with pair-wise combinations.Dr. Genichi Taguchi was one of the first proponents of orthogonal arrays in test design. His techniques are known as Taguchi Methods. Taguchi suggests five major steps in the designing process which take the test team from formulating the test problem to creating a good test design and refining the test design.  The steps are Formulate the problem, Plan the experiment, Analyze the results, Confirm the experiment and Adopt the new design. Taguchi’s approach focuses on the first two steps since they are very important. Taguchi methods provide an efficient and systematic way to optimize designs for performance, quality, and cost in automobiles and consumer electronics industries.Orthogonal Array Testing Strategy (OATS) is a systematic, statistical way of testing pair-wise interactions. It is based on creating Parameters (control Variables) and levels (values) for the parameters, which are the inputs to testable functions. This makes the technique particularly useful for flow based testing of software products. It is also quite useful for testing combinations of configurable options.

Test Suite Development Using OA:
Test case selection poses an interesting dilemma for the software professional. Executing a concise, well-defined set of tests that are likely to uncover most (not all) of the bugs give you a great deal more comfort in the quality of your software.So our objective is to design a test suite which maximizes the test coverage and optimize the no. of test cases and the effort required. Orthogonal array (OA) is a technique to achieve this objective. However, a good analysis is required before deciding on the usage of OA for test suite development with respect to identifying the parameters and levels. The basic fault model that lies beneath this technique is:
·    Interactions and integrations are a major source of defects. Most of these defects are not a result of complex interactions and most of these defects arise from simple pair-wise interactions.
·    Randomly selecting values to create all of the pair-wise combinations is bound to create inefficient test sets with random, senseless distribution of values and it is easy to miss one or more of the possible combinations.
OATS provides a means to select a Test Set that:
·    Guarantees testing the pair-wise combinations of all the selected variables by creating an efficient and concise test set with many fewer test cases than testing all combinations of all variables.
·    Guarantees an even distribution of all pair-wise combinations.
·    Exercises some of the complex combinations of all the variables.
·    Is simpler to generate and less error prone than test sets created by hand.

Steps to Use OATS
1.    Decide how many independent variables will be tested for interaction. This will map to the Factors of the array.
2.    Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array.
3.    Find a suitable orthogonal array with the smallest number of Runs. A suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2.
4.    Map the Factors and values onto the array.
5.    Choose values for any "left over" Levels.
6.    Transcribe the Runs into test cases, adding any particularly suspicious combinations that aren't generated.

illustration:
As an example, consider a system that has four options, each of which can have three values. The exhaustive test set would require 34 i.e. 81 test cases. The test set created by OATS has only nine test cases, yet tests all of the pair-wise combinations. 
·    The OATS test set is only 11% as large as the exhaustive set and will uncover most of the interaction bugs.
·    It covers 100% (9 of 9) of the pair-wise combinations, 33% (9 of 27) of the three-way combinations and 11% (9 of 81) of the four-way combinations.
·    The test set could easily be augmented if there were particularly suspicious three- and four-way combinations that should be tested. These test cases are called OA Complementary Coverage (OACC) test cases.

Limitation of OATS:
·    All the methodology is applicable only if the parameters identified are independent.
·    Can be effectively used if numbers of input parameters is small and the values that each of the parameters may take are clearly bounded.
·    For some problem sizes there does not exist enough orthogonal arrays to represent the entire problem

Conclusion:
It is beneficial to the Software product testing to make use of the Orthogonal Arrays. The only constraint would be the applicability of the concept to certain types of products. Software product companies can enjoy the benefits of OA which include Productivity Improvement, Less Implementation Time and High Code Coverage etc.

References:
Orthogonal Array Testing Strategy (OATS) Technique - Jeremy M. Harrell,
http://www.isixsigma.com/ ,
http://ist.psu.edu/courses/fa04/ist412/LectureNotes/Testing.pps,
http://ise.gmu.edu/~ofut/rsrch/papers/evalcombstrat.pdf