What is testing?

It is the process of comparing "what is" with "what ought to be".

The process of operating a system or component under specified conditions, observing or recording the results and making an evaluation of some aspect of the system or component.

Testing is a concurrent lifecycle process of engineering, using and maintaining test ware in order to measure and improve the quality of the software being tested.

Why do we test?

* To make a judgment about quality or acceptability.
* To discover problems.

Testing and Software Quality Assurance (SQA)

SQA is more concerned with reducing errors endemic in the development process. Testing is more concerned with discovering faults in a product.

A Testing Lifecycle

In the development phases, three opportunities arise for errors to be made resulting in faults that propagate through the remainder the development process.

1. The first three phases are putting bugs in
2. The testing phase is finding bugs
3. The last three phases are getting bugs out.

The fault resolution step is another opportunity for errors. When a fix causes formerly correct software to misbehave, the fix is deficient.

The process of testing can be subdivided into separate steps:

Test planning
Test case development
Running test cases
Evaluating test results

Venn Diagram to show relationship between the set T of test cases and the sets S and P of specified and implemented behaviors.

Program Behaviors

Consider a universe of program behaviors:

Set S of specified behaviors
Set P of programmed behaviors

The goal of tester is to make the region where these sets all intersect (region 1) as large as possible.

If specified behaviors exist for which no test case are available, the testing is necessarily incomplete.
If certain test cases correspond to unspecified behaviors, two possibility arise either such a test case is unwarranted or the specification is deficient.

Two fundamental approaches are used to identify test cases:

Functional Testing also called Black-box Testing
Structural Testing also called White-box Testing

Functional Testing

Program can be considered to be a function that maps values from its input domain to values in its output range.

The only information used to generate the test cases is the specification the software.

Because functional methods are based on the specified behavior, it is hard to imagine that they identify behaviors that are not specified.

Approaches include boundary value analysis, robustness testing, input/output equivalence classes, decision table, worst- case analysis.


They are independent of how the software is implemented, so if the implementation changes, the test cases are still useful.

Structural Testing

To understand structural testing familiarity with the concepts of linear graph theory is essential.

Uses information of how the function is actually implemented to generate test cases.

Structural methods are based on the program, it is hard to imagine these methods identify behaviors that are not programmed.

Approaches include various test coverage metrics like statement, condition, path etc.

Drawbacks of Functional Testing

Significant redundancies may exist among test cases.
Possibility of gaps of untested software.

Note: If all specified behaviors have not been implemented, structural test cases will never will able to recognize this. Conversely, if the program implements behaviors that have not been specified, this will never be revealed by functional test cases.