Software Testing

In spite of the fact that in every software development project the product undergoes testing, delivered software always contains residual defects. Software testing is a difficult, time-consuming process. It requires specific skills from software testers, skills that only partially overlap with those of software developers. Apart from mastering coding, testers must also possess a great deal of knowledge of formal languages, graph theory and algorithms.

• Modeling the software’s environment
• Selecting test scenarios
• Running and evaluating test scenarios
• Measuring testing progress

In the first phase, the tester’s task is to simulate the interaction between the application and its environment, be it the user or the other applications, taking into account all possible inputs and outputs that can cross the application’s boundaries. The hardest part here is the fact that in many cases the interactions can go through numerous different file formats, communication protocols, GUIs, and file systems. The other hard part is the unpredictability of the user’s actions the software under test must account for that. Since the number of possible test scenarios is usually extremely large, testers should select those scenarios that cover all code statements and all significant representatives of external events. Before running the selected scenarios, it is necessary to convert them into executable form (often as code) in order to
simulate typical interactions between the system and the external world. Applying test scenarios manually is labor-intensive and error-prone. For that reason, testers try to automate the test scenarios as much as possible. In many environments, automated application of inputs through code that simulates users is possible, and tools are available to help.

Measuring testing progress is difficult, simply because it is not just counting the numbers of bugs found. As stated in the section on software metrics, specific metrics for software testing are used to measure the coverage of the tests applied (in terms of running all lines of the source code, forcing all the internal data to be initialized and used, applying all test scenarios, exploring all the inputs, and checking for functional completeness). Note also that software reliability
engineering can greatly help - the Cleanroom methodology developed at IBM has been particularly useful in improving software quality and providing a quantitative measure for the quality of a software product at its release. TheCleanroom approach provides for the transition of process technology to the project staff and integrates several proven software-engineering practices into one methodology. The testing strategy of the Cleanroom methodology can
be best described as random sample based on usage model that predicts field reliability, rather than a futile attempt for coverage and little insight on field reliability.

If the Unified Process is used to manage software development, software testing is performed in every iteration. Test scenarios are defined from use cases, and comprise both functionality and performance testing. The advantage of this incremental and iterative approach to software testing is that in each iteration the testers test just some of the application. Moreover, the tests performed in early phases usually discover such bugs and faults that would
cause more severe instability in the project’s rhythm if they were discovered in later phases. In every iteration, tests also check whether the current iteration has jeopardized some of the previously built and tested architecture. If the project’s size is large, it is impractical to manually run all the test cases, so the use of automated testing tools is recommended. Project managers should adopt the practice of enforcing thorough testing in every iteration, and not allowing the
next iteration to begin before all the tests planned in the current iteration are completed. The entire project is considered completed only when all the UML models and all the tests are completed and delivered.