Testing Principles in Software Testing

Testing Principles in Software Testing

Performing Testing has always been a crucial and challenging task. Thus, to perform testing effectively and efficiently, various guild lines have been presented and suggested by scholars already in the field. Numerous testing principles have been accumulated over the period of last few decades by rigorous understanding of the testing psychology.

Here, in this blog set, we would discuss in detail about these testing principles and how are they applicable in the real life scenarios while performing testing.

Before kick starting, Let us have a quick look at what can be the potential threats/problems which demands the need of testing.

•         Failures can be caused due to errors in specification, design and implementation of the software.
•         Errors in use of the system
•         Environmental conditions
•         Intentional damage
•         Potential Consequences of earlier errors, defects and failures.

Now, when we know that testing activities are an inherent need of the software development process in order to achieve functional and non- functional quality aspects, let us discuss the testing principles.

Testing Principle 1: Testing is context dependent

Testing is done differently in different contexts. The type of testing applicable for “Problem-A” may not fit to the “Problem-B”.

Testing shall always be done differently in different context. Generally, it depends on the level of risk and impact associated with the work product. Occasionally, it may depend on the factors like deadlines, time pressure, resource availability and market needs.

A risk is something that has not yet happened but has a probable happening; it is a potential problem. When we discuss risks, we shall also consider the impact of the risk if it happens.

If these risks occur, their solution may be costlier and damaging. Therefore, risk analysis should be done prior to testing and thus accordingly testing technique and approach shall be chosen within the context.

We always have certain questions, how much testing should be done? At what time can we say that the system is fully tested? Can testing be completely done? To answer these questions we have our next testing principle.

Testing Principle 2: Exhaustive testing is impossible

Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts.

Choice is ours:

•       Test Everything
•       Test Nothing
•       Test some part of the software

Ideally, we shall test everything, but practically this seems impossible. Let us take an example. If a 1 digit field is to be tested, then there can be “n” no. of test cases with 26 lower case, 26 upper case, 0-9 numeric values, random junk characters and 6 special characters. Thus, leading to devise an approach which does the right amount of testing, considering the time pressure and the project budget.

Test some part of the software

Perform risk assessment to comprehend how much testing needs to be done. The efforts that need to be put into testing activities are more or less tailored by risks and cost associated with the project.

The next testing principle answers questions like: When can we start testing?

Testing Principle 3: Early Testing

Testing activities should start as early as possible in the software or system development life cycle.

Early testing – such as early test design and review activities – finds defects early on when they are cheap to find and fix.

Defect (Requirements + Design) = ½ Defect (Total)

Defect Propagation:

Defect (Requirements) ——-> Propagates at various stages (Design & Code)

Cost of Defects:

Defect Cost (fix & finding) : Life Cycle Stages (Specification < Design < Build < Test)

Considering the above factors, it is highly recommended to start testing in the project as soon as it is possible.

Testing Principle 4: Defect Clustering

A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures.

There might be areas in the project, which can be tricky and complex, thus these are the hot spots of the application for testing as they tends to exhibit the maximum number of defects. Therefore, it is an important task to identify the potential clusters.

Testing Principle 5: Pesticide Paradox

If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this ‘pesticide paradox’, the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

If we have been testing for a long while with the similar set of test cases, it tends to decrease the number of defects as we move forward. After a certain stage, the same set of test cases does not find the defects anymore. Thus we might need to change the set of test cases with another set of test cases in accordance with the next set of risks to have more defects exposed.

The next testing principle answers questions like: Can i say that the software is defect free?

Testing Principle 6: Testing shows presence of defects

Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software.

Even after thoroughly testing a product we can’t say that product is 100% defect free.Testing can only reduce the number of defects/Bugs and also the risk associated with the product but can’t ensure that product is bug free.

 

Testing Principle 7: Absence of errors fallacy
if some set of test cases/Scenarios are not able to find the defect in a product doesn’t prove that product is bug free and further user can’t find bug in system/product.

 

Above a brief to testing principles has been given, but many of these principles would already be in use within your project. Thus the blog gives a fair idea of the keys factors to testing.