Fundamental Test Process

The fundamental test process comprises planning, specification, execution, recording and checking for completion. In more detail:

• Test planning: The test plan should specify how the test strategy and project test plan apply to the software under test. This should include identification of all exceptions to the test strategy and of all software with which the software under test will interact during test execution, such as drivers and stubs.
• Test specification: Test cases should be designed using the test case design techniques selected in the test planning activity.
• Test execution: Each test case should be executed.
• Test recording: The test records for each test case should unambiguously record the identities and versions of the software under test and the test specification. The actual outcome should be recorded. It should be possible to establish that all of the specified testing activities have been carried out by reference to the test records.
  
  The actual outcome should be compared against the expected outcome. Any discrepancy found should be logged and analysed in order to establish where its cause lies and the earliest test activity that should be repeated, e.g. in order to remove the fault in the test specification or to verify the removal of the fault in the software.
  
  The test coverage levels achieved for those measures specified as test completion criteria should be recorded.
• Checking for test completion:
  The test records should be checked against the previously specified test completion criteria. If these criteria are not met, the earliest test activity that must be repeated in order to meet the criteria should be identified and the test process should be restarted from that point.
  
  It may be necessary to repeat the Test Specification activity to design further test cases to meet a test coverage target.

As the objective of a test should be to detect faults, a 'successful' test is one that does detect a fault. This is counter-intuitive, because faults delay progress: a successful test is one that may cause delay. The successful test reveals a fault which, if found later, may be many times more costly to correct so, in the long run, is a good thing.

Completion or exit criteria are used to determine when testing (at any test stage) is complete. These criteria may be defined in terms of cost, time, faults found or coverage criteria. Coverage criteria are defined in terms of items that are exercised by test suites, such as branches, user requirements, most frequently used transactions, etc.