Activity: Execute Test

Purpose To execute tests and capture test results.
Steps Execute Test Procedures Evaluate Execution of Test Verify Test Results Recover From Halted Tests
Input Artifacts: Components Development Infrastructure Implementation Subsystems Software Build Test Cases Test Model Test Procedures Test Scripts Test Component Test Subsystem	Resulting Artifacts: Test Results
Worker: Tester

Workflow Details:

Core Workflow: Implementation

Integrate Each Subsystem

Integrate the System

Core Workflow: Test

Execute Tests in Integration Test Stage

Execute Tests in System Test Stage

Core Workflow: Deployment

Manage Acceptance Test

Core Workflow: Configuration & Change Management

Manage Change Requests

Execute Test Procedures

Purpose

To execute the test procedures (or test scripts if testing is automated)

Tool Mentors:

Rational Robot^TM

Rational TestFactory^TM

Executing a Suite of Test Scripts Using Rational TestFactory

Rational LoadTest^TM

Executing a Performance Test Schedule

To execute the tests, the following steps should be followed:

Set-up the test environment to ensure that all the needed components (hardware, software, tools, data, etc.) have been implemented and are in the test environment.

Initialize the test environment to ensure all components are in the correct initial state for the start of testing.

Execute the test procedures.

Note: executing the test procedures will vary dependent upon whether testing is automated or manual.

Automated testing: The test scripts created during the Implement Test activity are executed.

Manual execution: The structured test procedures developed during the Design Test activity are used to manually execute test.

Evaluate Execution of Test

Purpose

To determine whether testing executed to completion or halted
To determine if corrective action is required

Tool Mentors:

Rational LogViewer^TM:

See section "Evaluate Execution of Test" in Tool Mentor: Evaluating the Execution of Test Using Rational LogViewer

Rational TestFactory:

Rational LoadTest:

Monitoring a Performance Test Schedule

The execution of testing ends or terminates in one of two conditions:

Normal: all the test procedures (or scripts) execute as intended and to completion.

If testing terminates normally, then continue with Verify Test Results:

Abnormal or premature: the test procedures (or scripts) did not execute completely or as intended. When testing ends abnormally, the test results may be unreliable. The cause of the abnormal / premature termination needs to be identified, corrected, and the tests re-executed before any additional test activities are performed.

If testing terminates abnormally, continue with Recover From Halted Tests.

Verify Test Results

Purpose

To determine if the test results are reliable
To identify appropriate corrective action the test results indicate flaws in the test effort or artifacts

Tool Mentors:

See sections "Verify Test Results" and "Investigate Unexpected Results" in Tool Mentor: Evaluating the Execution of Test using Rational LogViewer

Upon the completion of testing, the test results should be reviewed to ensure that the test results are reliable and reported failures, warnings, or unexpected results were not caused by external influences (to the target-of-test), such as improper set-up or data.

The most common failures reported when test procedures and test scripts execute completely, and their corrective actions are given below:

Test verification failures - this occurs when the actual result and the expected result do not match. Verify that the verification method(s) used focus only on the essential items and / or properties and modify if necessary.
Unexpected GUI windows - this occurs for several reasons. The most common is when a GUI window other than the expected one is active or the number of displayed GUI windows is greater than expected. Ensure that the test environment has been set-up and initialized as intended for proper test execution.
Missing GUI windows - this failure is noted when a GUI window is expected to be available (but not necessarily active) and is not. Ensure that the test environment has been set-up and initialized as intended for proper test execution. Verify that the actual missing windows are / were removed from the target-of-test.

If the reported failures are due to errors identified in the test artifacts, or due to problems with the test environment, the appropriate corrective action should be taken and the testing re-executed. For additional information, see "Recover From Halted Tests" below.

If the test results indicate the failures are genuinely due to the target-of-test, then the Execute Test Activity is complete.

Recover From Halted Tests

Purpose

To determine the appropriate corrective action to recover from a halted test
To correct the problem, recover, and re-execute the tests

Tool Mentors:

See section "Recover From Halted Tests" in Tool Mentor: Evaluating the Execution of Test using Rational LogViewer

There are two major types of halted tests:

Fatal errors - the system fails (network failures, hardware crashes, etc.)
Test Script Command Failures - specific to automated testing, this is when a test script cannot execute a command (or line of code).

Both types of abnormal termination to testing may exhibit the same symptoms:

many unexpected actions, windows, or events occur while the test script is executing
test environment appears unresponsive or in an undesirable state (such as hung or crashed).

To recover from halted tests, do the following:

determine the actual cause of the problem
correct the problem
re-set-up test environment
re-initialize test environment
re-execute tests

Rational Unified Process