Software Testing Principles

    SEVEN TESTING PRINCIPLES, as specified by ISTQB® Certified Tester Foundation Level Syllabus, are explained below:

    1. Testing shows the presence of defects, not their absence

    This correlates to the saying ‘All code is guilty until proven innocent’. Paradoxically, the proving of ‘innocence’ of code is technically impossible (See principle 2 below) and we can only prove if some code is ‘guilty’ when we give the evidence of a defect in it.

    2. Exhaustive testing is impossible

    Since there is an infinite number of scenarios, environments and paths, it is technically impossible to test all of them. Hence, it is essential that we prioritize testing based on business impact, critical features, frequently used functionalities, complex implementation and buggy areas of the software. Prioritization can also be based on what changes have been made in the version of the software and the likely areas in the software that might have been impacted due those changes.

    3. Early testing saves time and money

    Conducting static testing (review) early, say during the requirements development phase, and dynamic testing early, say unit testing, requires some level of early investment but can save a lot of time and effort in the long run. For example, if a requirement defect is caught right away during requirement review, it can be easily fixed by correcting the error in the specifications document. But, imagine the waste of time if the defect was caught only during acceptance testing: time is spent on analyzing the defect, finding its root cause, fixing the defect, confirming the fix, running a regression test, re-releasing the software, etc (And, oh yes, there’s also the trouble of finding the person to put the blame on.)

    4. Defects cluster together

    The 80-20 rule, also known as the Pareto Principle, which asserts that 80% of outcomes (or outputs) result from 20% of all causes (or inputs) for any given event holds true in software development too. Hence, in order to maximize the effectiveness of testing, we need to identify areas which are prone to defects and focus our testing in those areas. Those areas are normally governed by complexity and interdependence.

    5. Beware of the pesticide paradox

    If we use the same pesticide time and again, its effectiveness in killing insects degrades over time. Why? The insects evolve. If we use the same tests time and again, its effectiveness in finding defects degrades over time. Why? The software evolves. Of course, those old tests may uncover some regression defects but they will not be able to detect new defects. When we see all those tests passing, it gives a false confidence in the quality of software. Hence, the target should be to add or enhance tests for every change in code.

    6. Testing is context dependent

    Just the way you use a screwdriver for fastening screws and a wrench for fastening bolts, you must change your tool as the purpose changes. Similarly, you must change your testing strategy depending on the nature of the software you are testing. For example, you cannot use the same methods and types of testing on a Rocket Launcher app as you used on a Ludo app; it may not be as simple as just roll & race!

    7. Absence-of-errors is a fallacy

    This principle correlates with principles 2 (Exhaustive testing is impossible) and 1 (Testing shows the presence of defects, not their absence). Assuming that software with a lack of defects is of high quality is a misconception. First of all, some defects may have been left undetected. But, even if we assume that all defects were detected and fixed, it does not necessarily mean that the software fully meets the users’ needs and expectations. Maybe the button works exactly as per the requirement specification and has no defect whatsoever but to the user it could look ugly. In order to understand this concept better, read our articles on Software Quality and Software Quality Dimensions.

    Last Updated on September 17, 2020 by STF