Why is testing necessary?

Causes of Software Defects

Time pressure

Human fallibility

Inexperienced or insufficiently skilled project participants

Miscommunication between project participants

The complexity of the code, design, architecture, problem etc

New, unfamiliar technologies

What is Testing?

Test planning

Analyzing, designing, and implementing tests

Reporting test progress and results

Evaluating the quality of a test object

Reporting on the testing process and system under test

Finalizing closure activities after a test phase has been completed

Reviewing documents (including source code) and conducting static analysis

To evaluate work products such as requirements, user stories, design, and code

To verify whether all specified requirements have been fulfilled

To validate whether the test object is complete and works as the users and other stakeholders expect

To build confidence in the level of quality of the test object

To prevent defects

To prevent defects

etc

Testing and Quality

Seven Testing Principles

1. Testing shows the presence of defects, not their absence

2. Exhaustive testing is impossible

3. Early testing saves time and money

4. Defects cluster together

5. Beware of the pesticide paradox

6. Testing is context dependent

7. Absence-of-errors is a fallacy

Fundamental Test Process

Test planning and control

Test analysis and design

Test implementation and execution

Test completion activities

Test Planning and Control

Test Analysis and Design

Analyzing the test basis appropriate to the test level being considered

Evaluating the test basis and test items to identify defects of various types, such as ambiguities, omissions, inconsistencies, inaccuracies, contradictions, superfluous statements

Identify and prioritize test conditions

Capturing bi-directional traceability between elements of the test basis and the associated test conditions

Designing and prioritizing test cases and sets of test cases

Identifying necessary test data to support test conditions and test cases

Designing the test environment and identifying any required infrastructure and tools

Capturing bi-directional traceability between the test basis, test conditions, test cases, and test procedures

Test Implementation and Execution

Developing and prioritizing test procedures, and, potentially, creating automated test scripts

Creating test suites from the test procedures and (if any) automated test scripts

Arranging the test suites within a test execution schedule in a way that results in efficient test execution

Building the test environment and verifying that everything needed has been set up correctly

Preparing test data

Verifying and updating bi-directional traceability between the test basis, test conditions, test cases, test procedures, and test suites

Test Completion Activities

Checking whether all defect reports are closed, entering change requests or product backlog items for any defects that remain unresolved at the end of test execution

Creating a test summary report to be communicated to stakeholders

Finalizing and archiving the test environment, the test data, the test infrastructure, and other testware for later reuse

Handing over the testware to the maintenance organization

Analyzing lessons learned from the completed test activities to determine changes needed for future iterations, releases, and projects

Using the information gathered to improve test process maturity

Summary

Testing is an important part of a software development. Testing is tightly coupled with development and many teams include not only developers but also testers (or even whole testing department). Because of this tight coupling, it is nice when testers and developers can share a common dictionary. With this goal, I'm posting here my notes on this topic.In this post, I will try to highlight what is testing, how it's connected with quality assurance and quality management. And also will describe seven testing principles and will path you through a fundamental test process.Testing of components, systems, and their associated documentation, can help reduce the risk of failures occurring during operations. When defects are detected subsequently fixed, this contributes to the quality of the components or systems.A human being can make an error (mistake), which produces a defect (fault, bug) in the program code, or in a document. If a defect in code is executed, the system may fail to do what it should do (or do something it shouldn’t), causing a failure.Defects in software, systems or documents may result in failures, but not all defects do so. A defect in the code, being executed, may cause failure, but not necessarily in all circumstances. For example, some defects require very specific inputs or preconditions to trigger a failure, which may occur rarely or never.Errors (mistakes, defects) may occur for many reasons, such as:In addition to failures caused due to defects in the code, failures can also be caused by environmental conditions.Not all unexpected test results are failures. False positive may occur due to errors in tests, test data, test environment or for other reasons. The same things can lead to false negative. False negatives are tests that do not detect defects that they should have detected; false positives are reported as defects, but aren’t actually defects.A common misperception of testing is that it only consists of running tests. software testing is a process which includes many different activities; test execution is only one of these activities. The test process also includes activities such as:Another common misperception of testing is that it focuses entirely on verification of requirements, user stories, or other specifications. While testing does involve checking whether the system meets specified requirements, it also involves validation, which is checking whether the system will meet user and other stakeholder needs in its operational environment(s).For any given project, the objectives of testing may include:Also good to note that testing and debugging are different. Executing tests can show failures that are caused by defects in the software. Debugging is the development activity that finds, analyzes, and fixes such defects.With help of testing, it is possible to measure the quality of software in terms of defects found, for both functional and non-functional software requirements and characteristics.Testing can give confidence in the quality of the software if it finds few or no defects. A properly designed test that passes reduces the risk of defects within the system.People often use phrase quality assurance (or just QA) to refer to testing, but they are not the same (so they are related). Quality management is the larger concept that ties them together. Quality management includes all activities that direct and control an organization with regard to quality and also includes such activities as quality assurance and quality control.Quality assurance is focused on adherence to proper processes, in order to provide confidence in the appropriate level of quality. When processes are carried out properly, the work products are generally of higher quality, which contributes to defect prevention.Quality control involves various activities, including test activities, that support the achievement of appropriate levels of quality. Since quality assurance is concerned with the proper execution of the entire process, quality assurance supports proper testing.Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but it is not a proof of correctness.Testing everything is not feasible. Instead, risk analysis, test techniques, and priorities should be used to focus test efforts.To find defects early, test activities should be started as early as possible in the software development lifecycle. Early testing is sometimes referred to as shift left.A small number of modules usually contains most of the defects or is responsible for most of the operational failures. Predicted defect clusters and the actually observed defect clusters in test or operation are an important input into a risk analysis used to focus the test effort (as mentioned in principle 2).If the same tests are repeated over and over again, eventually these tests no longer find any new defects. To detect new defects, existing tests and test data may need changing, and new tests may need to be written. (Tests are no longer effective at finding defects, just as pesticides are no longer effective at killing insects after a while.)Testing is done differently in different contexts. For example, safety-critical industrial control software is tested differently from an e-commerce mobile app.Some organizations expect that testers can run all possible tests and find all possible defects, but principles 2 and 1, respectively, tell us that this is impossible. Further, it is a fallacy (i.e., a mistaken belief) to expect that just finding and fixing a large number of defects will ensure the success of a system. For example, fully testing all specified requirements and fixing all defects found could still produce a system that is difficult to use, that does not fulfill the users' needs and expectations.A test process consists of the following main groups of activities:Test planning is the act of defining the objectives of testing and specification of test activities in order to meet test objectives. On the other hand, control is the act of comparing actual progress against the test plan and reporting the status including deviations from the plan. Test planning takes into account the feedback from monitoring and control activities.Test analysis determines “what to test” in terms of measurable coveragecriteria and includes next major activities:So, test analysis answers the question “what to test?” while test designanswers the question “how to test?" and includes next major activities:So, test design answers the question “how to test?” while test implementation answers the question “do we now have everything in place to run the tests?”.Test implementation includes the following major activities:Test completion activities collect data from completed test activities to consolidate experience and any other relevant information. Test completion activities occur at project milestones such as when a software system is released, a test project is completed, an Agile project iteration is finished etc.Test completion includes the following major activities:In this post, I've tried to cover the basics of testing in software development. We've learned what is testing, how it's connected with quality assurance and quality management. Also, we've learned seven testing principles and went through a fundamental test process. I hope you liked it. Thank you for reading!