software testing

What is software testing?

Software testing is a process of verifying and validating that a software application or program

1. Meets the business and technical requirements that guided its design and development, and

2. Works as expected.

Software testing also identifies important defects, flaws, or errors in the application code that must be fixed. The iterative process of software testing consists of:

Ø Designing tests

Ø Executing tests

Ø Identifying problems

Ø Getting problems fixed

Software testing has three main purposes : verification ,validation , defect finding

Ø The verification process confirms that the software meets its technical specifications . Through verification we make sure the product behaves the way we want it to .Verification techniques include requirements review,design review,code walkthrough , and code inspection

Ø Through validation , we check to make sure that in the process a mistake hasn’t been made such that the product build is not what the customer asked for , validation always involves comparison against requirements . Validation techniques include unit testing , integration testing , system testing and user acceptance testing.

Ø A defect is a variance between the expected and actual result The defect’s ultimate source may be traced to a fault introduced in the specification , design , or development (coding) phases .

Need of software testing

Testing during development phase includes debugging and code review .Software testing answers questions that development testing and code reviews cant

Ø Does it really work as expected

Ø Does it meet the user’s requirements

Ø Is it what the user’s expect

Ø Do the user’s like it

Ø Is it compatible with other systems

Ø How does it perform

Ø How does it scale when more users are added

Ø Which areas need more work

Ø Is it ready for release

What can we do with answers to these questions

Ø Save time and money by identifying defects early

Ø Avoid or reduce development downtime

Ø Provide better customer service by building a better application

Ø Know that we have satisfied our user’s requirements

Ø Build a list of desired modifications and enhancements for later versions

Ø Identify and catalog reusable modules and components

Ø Identify areas where programmers and developers need training

What do we test

First we test what is important by focusing on the core functionality – the parts that are critical or popular before looking at the ‘nice to have features’.Concentrate on the application’s capabilities in common usage situations before going on to unlikely situations.For e.g. if application retrieves data and performance is important , test reasonable queries with normal load on server before going on to unlikely ones at peak usage times . The value of software testing goes far beyond testing underlying code .It also examines functional behavior of the application .Its possible that the code is solid but the requirements were inaccurately or not completely communicated .Testing can involve some or all of the following factors .The more, the better

Ø Business requirements

Ø Functional design requirements

Ø Technical design requirements

Ø Regulatory requirements

Ø Programmer code

Ø Systems administration standards and restrictions

Ø Corporate standards

Ø Professional or trade association best practices

Ø Hardware configuration

Ø Cultural issues and language differences

Who does the testing

Software testing is not one person job .It takes a team of testers depending on the size and complexity of applicaton being tested . Generally developers should have reduced role in testing as they may not be able to take an unbiased look at the results of their labors . Testers must be cautious , curious , critical but non-judgmental and good communicators . Testers are the only IT people who will use the system as heavily an expert user on the business side . Testers must find the defects that only expert users will find because experts may not report problems if they find.

Key players and their roles

Business Sponsor(s) and partners

Ø Provides funding

Ø Specifies requirements and deliverables

Ø Approves changes and some test results

Project Manager

Ø Plans and manages the project

Software developer(s)

Ø Designs,codes and builds the application

Ø Participates in code reviews and testing

Ø Fixes bugs ,defects and shortcomings

Testing Coordinator(s)

Ø Creates test plans and test specifications based on requirements and functional, and technical documents Tester(s)

Ø Executes the tests and documents results

Verification Phases

Requirement analysis -In this phase, the requirements of the proposed system are collected by analyzing the needs of the user(s). This phase is concerned about establishing what the ideal system has to perform. However, it does not determine how the software will be designed or built. Usually, the users are interviewed and a document called the user requirements document is generated. The user requirements document will typically describe the system’s functional, physical, interface, performance, data, security requirements etc as expected by the user. It is one which the b usiness analysts use to communicate their understanding of the system back to the users. The users carefully review this document as this document would serve as the guideline for the system designers in the system design phase. The user acceptance tests are designed in this phase.

System design- System engineers analyze and understand the business of the proposed system by studying the user requirements document. They figure out possibilities and techniques by which the user requirements can be implemented. If any of the requirements are not feasible, the user is informed of the issue. A resolution is found and the user requirement document is edited accordingly.

The software specification document which serves as a blueprint for the development phase is generated. This document contains the general system organization, menu structures, data structures etc. It may also hold example business scenarios, sample windows, reports for the better understanding. Other technical documentation like entity diagrams, data dictionary will also be produced in this phase. The documents for system testing is prepared in this phase.

Architecture design -This phase can also be called as high-level design. The baseline in selecting the architecture is that it should realize all the requirements within the given time, cost and resources. Software architecture is commonly represented as two-tier, three-tier or multi-tier models which typically consist of the database layer, user-interface layer and the application layer. The modules and components representing each layer, their inter-relationships, subsystems, operating environment and interfaces are laid out in detail.

The output of this phase is the high-level design document which typically consists of the list of modules, brief functionality of each module, their interface relationships, dependencies, database tables, architecture diagrams, technology details etc. The integration testing design is carried out in this phase.

Module Design - This phase can also be called as low-level design. The designed system is broken up in to smaller units or modules and each of them is explained so that the programmer can start coding directly. The low level design document or program specifications will contain a detailed functional logic of the module, in pseudocode- database tables, with all elements, including their type and size - all interface details with complete API references- all dependency issues- error listings- complete input and outputs for a module. The unit test design is developed in this stage.

Coding – In this phase the programmers write the actual source code of the application being developed in programming language . They also perform testing , debugging / troubleshooting and maintaining the source code

Validation Phases

Unit Testing - the V-model of software development, unit testing implies the first stage of dynamic testing process. A fault discovered and corrected in the unit testing phase is more than a hundred times cheaper than if it is done after delivery to the customer.It involves analysis of the written code with the intention of eliminating errors. It also verifies that the codes are efficient and adheres to the adopted coding standards. Testing is usually white box. It is done using the Unit test design prepared during the module design phase. This may be carried out by software testers, software developers or both.

Integation Testing - In integration testing the separate modules will be tested together to expose faults in the interfaces and in the interaction between integrated components. Testing is usually black box as the code is not directly checked for errors. It is done using the integration test design prepared during the architecture design phase. Integration testing is generally conducted by software testers.

System Testing - System testing will compare the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. Once all the modules are integrated several errors may arise. Testing done at this stage is called system testing.

User Acceptance Testing - Acceptance Testing checks the system against the requirements of the user. It uses black box testing using real data, real people and real documents to ensure ease of use and functionality of systems. Users who understand the business functions run the tests as given in the acceptance test plans, including installation and Online help. Hard copies of user documentation are also being reviewed for usability and accuracy. The testers formally document the results of each test, and provide error reports, correction requests to the developers.

The Basics of Software Testing

There are two basic techniques of software testing – black box testing and white box testing

Black Box Testing (also called functional testing) is testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions . In black box testing the tester does not have direct access to source code .The code is considered to be a “black box” to the tester who cant see inside the box . The tester knows only that informaion can be input into the black box and black box will send some output.Based on requirements knowledge ,tester knows what to expect the black box to send out and tests to make sure that the output is what is expected.In the language of V&V , black box testing is often used for validation (are we building the right software?)

Strategies for Black Box Testing

Tests of Customer Requirements – Black box test cases are based on customer requirement. To start with we make sure that every single requirement has been tested at least once. The first test case we write for any given requirement is the most used success-path for that requirement i.e we want to execute some desirable functionality without any error conditions . Then we proceed with test cases that execute failure-path. Failure paths intentionally have some kind of errors that users can accidentally input.We must make sure that program behaves predictably and gracefully in the face of these errors.Finally we should plan the execution of out tests out such that most troublesome ,risky requirements are tested first .This would allow more time for fixing problems before delivering the product to the customer.

Equivalence Partitioning – Equivalence partitioning is a method for deriving test cases. In this method, classes of input conditions called equivalence classes are identified such that each member of the class causes the same kind of processing and output to occur. In this method, the tester identifies various equivalence classes for partitioning. A class is a set of input conditions that are is likely to be handled the same way by the system. If the system were to handle one case in the class erroneously, it would handle all cases erroneously. Equivalence partitioning drastically cuts down the number of test cases required to test a system reasonably. It is an attempt to get a good 'hit rate', to find the most errors with the smallest number of test cases. To use equivalence partitioning, you will need to perform two steps

Ø Identify the equivalence classes

Ø Design test cases

Take each input condition described in the specification and derive at least two equivalence classes for it. One class represents the set of cases which satisfy the condition (the valid class) and one represents cases which do not (the invalid class )For example, if an item in inventory can have a quantity of - 9999 to + 9999, identify the following classes:

Ø one valid class: (QTY is greater than or equal to -9999 and is less than or equal to 9999).This is written as(- 9999 < = QTY < = 9999)

Ø the invalid class (QTY is less than -9999), also written as (QTY < -9999)

Ø the invalid class (QTY is greater than 9999) , also written as (QTY >9999)

Boundary Value Analysis – Programers often make mistake on the boundaries of equivalent classes/input domain.As a result we need to focus testing at these boundaries .This type of testing is called Boundary Value Analysis (BVA). Boundary value analysis is a methodology for designing test cases that concentrates software testing effort on cases near the limits of valid ranges Boundary value analysis is a method which refines equivalence partitioning. Boundary value analysis generates test cases that highlight errors better than equivalence partitioning. The trick is to concentrate software testing efforts at the extreme ends of the equivalence classes. At those points when input values change from valid to invalid errors are most likely to occur. As well, boundary value analysis broadens the portions of the business requirement document used to generate tests.Unlike equivalence partitioning, it takes into account the output specifications when deriving test cases.For example, if the output specifications for the inventory system stated that a report on inventory should indicate a total quantity for all products no greater than 999,999, then we add the following classes

Ø The valid class ( 0 < = total quantity on hand < = 999,999 )

Ø The invalid class (total quantity on hand <0)

Ø The invalid class (total quantity on hand> 999,999 )

Failure (“Dirty”)Test Cases – In this technique we think of every possible thing a user could possibly do with the system to demolish the software. We need to make sure your program is robust – in that

it can properly respond in the face of erroneous user input. This type of testing is called robustness testing, whereby test cases are chosen outside the domain to test robustness to unexpected, erroneous input, and is included in defensive testing which includes tests under both normal and abnormal conditions

Look at every input. Does the program respond “gracefully” to these error conditions?

1. Can any form of input to the program cause division by zero?

2. What if the input type is wrong? (You’re expecting an integer, they input a float. You’re expecting a character, you get an integer.)

3. What if the customer takes an illogical path through your functionality?

4. What if mandatory fields are not entered?

5. What if the program is aborted abruptly or input or output devices are unplugged?

Test Early and often - Executing your test cases as soon as possible is an excellent way of getting concrete feedback about your program. In order to run test cases early, programmers need to integrate the pieces of their code into the code base often. Programmers could be tempted to work on their own computer until the finish implementing a “whole” requirement. In industry, this could quite feasibly mean they keep their code to themselves for several months. However, this is a dangerous practice. Just because a component works on a programmer’s own computer, this doesn’t mean it will work when it is assembled with the code other programmers are working on. The earlier it is known that there are some interface problems or some data that’s not getting passed properly the better. This knowledge can only be gained by integrating code and testing early and often. Then, integration problems can be more easily localized in the work that was just integrated. By localizing the code that contains a new defect, the programmer can efficiently identify and remove defects.

White Box Testing (also called as structural/clear box/glass box) is testing that takes into account the internal mechanism of the system or component.The connotations of “clear box” and “glass box” appropriately indicate that you have full visibility of internal workings of the software product,specifically ,the logic and the structure of the code .Using the white-box testing techniques ,a software engineer can design test cases that (1)exercise independent paths within a module or unit (2)exercise logical decisions on both their true and false side (3)execute loops at their boundaries and within their operational bounds and (4)exercise internal data structures to ensure their validity

White box testing using stubs and drivers

With white box testing the code must be run with predetermined input and check to make sure that code produces predetermined outputs .Often programmers write stubs and drivers for white box testing . A driver is a software module used to invoke a module under test and often provide test inputs ,control and monitor execution and report test results . e.g. if you want to move a player on the board , the driver code would be

MovePlayer (Player,diceRoll);

This driver code would likely be called from main method .A white box test case would execute this driver line of code and check Player.getPosition()to make sure the player is now on the expected cell on the board

A stub is a computer program statement substituting for the body of a software module that is or will be defined elsewhere or a dummy component or object used to simulate the behavior of a real component until that component has been developed . If movePlayer method has not been written yet ,a stub such as one below maybe used temporarily – which moves any player to position 1

Public void movePlayer(Player player , int diceValue){

player.setPosition(1);

}

Ultimately ,the dummy method would be completed with proper program logic.However, developing the stub allows programmer to call a method in the code being developed even if the method does not yet have the desired behavior.