Tool Mentor: Creating a Performance Test Schedule to Represent a Performance Test Workload

Purpose

This tool mentor describes how to design a schedule to represent a workload for performance testing using Rational LoadTest. You will get more realistic results from the tests if you properly emulate the workload.

Related Rational Unified Process activities:

Section "Record, Generate, or Program Test Scripts" of Activity: Implement Test
Section "Workload Analysis" of Activity: Design Test
Section "Identify and Describe Test Cases" of Activity: Design Test

Also see the following for additional related information:

Overview

When doing performance testing with LoadTest, you want to emulate your actual work environment. If you are testing performance, it is essential that your workload model mirror the workload at your site. Therefore, you must determine the types of transactions that occur at your site. For example, do your users generally query the database and update it occasionally, or do they update it frequently? If they update the database frequently, are the updates complex and lengthy or are they short?

Here are some factors to consider when designing the workload:

The workload interval ù The period of time the workload is supposed to represent. For example, the workload interval could be a peak hour, an average day, or an end-of-month billing cycle.

Test variables ù The factors you will change during the performance test. For example, you might vary the number of users to understand how response time degrades as the workload increases.

It is best to change only one variable at a time. Then if performance changes, you know that the change was caused by that variable. Test variables are set when you set up a schedule.

Functional user classifications (user groups) ù Categorize the users into groups based on the types of activities they perform. For each user group, you need to identify the number of users or the percentage of overall users. For example, you might group 20% of users into Accounting, 30% into Data Entry, and 50% of the users into Sales.

You set up user groups in a schedule. However, you should plan your scripts beforehand so that when you record them they reflect the actions of a user group.

User work profiles ù The set of activities that the users perform and the frequency with which they perform them. The user actions should mirror as closely as possible the mix of tasks that the users actually perform. For example, if the Sales user group accesses the database 70% more than the other two groups, be sure the workload reflects this.

User characteristics ù How long a user pauses before executing a transaction, the typing rates, and the number of times a transaction is executed consecutively. It is important to model the user characteristics accurately because the values directly affect the overall performance of the system. For example, a user who thinks for 5 seconds and types 30 words per minute puts a much smaller workload on the system than a user who thinks for 1 second and types 60 words per minute. You use delays and think times to model the user characteristics.

To design a schedule to represent a workload:

Insert user groups.
Insert scripts.
Insert other items.
Set the order in which the schedule items will run.

1. Insert user groups.

A user group is the basic building block for all schedules. A user group is a collection of users that perform the same activity. For example, a schedule may contain these three user groups: Accounting, Data Entry, and Sales. Each user group has its own characteristics and affects the workload in different ways.

The procedure for inserting user groups into a schedule is found in Chapter 7 of the Using Rational LoadTest manual, which can be found on the documentation CD.

2. Insert scripts.

After you insert user groups into the schedule, you add the scripts that the user groups run. For example, the Accounting group may run two scripts, the Data Entry group may run five scripts, and the Sales group may run three scripts. The scripts contain activities that a particular group of users will perform. For example, one of the scripts the Accounting group may perform is to look up the payment status for an invoice.

The procedure for inserting scripts into a schedule is found in Chapter 7 of the Using Rational LoadTest manual, which can be found on the documentation CD.

3. Insert other items.

A schedule requires only user groups and scripts to run. However, a schedule that realistically models the work that actual users perform is likely to contain other items as well.

The following items can be added to a schedule:

Scenarios
Executables
Delays
Synchronization points
Transactors

Scenarios
A scenario lets you group scripts together so they can be shared by more than one user group. If you have a complicated schedule that uses many scripts, grouping the scripts under a scenario has the added advantage of making your schedule easier to read and maintain.

You define a scenario in the Scenarios section of the schedule by inserting a scenario and then inserting items within it. To make a user group execute the scenario, you must insert the scenario name in a user group.

Executables
An executable is a program, such as Notepad or Excel, that runs on Windows NT. You can insert an executable into a GUI user group only. When you run the schedule, the executable runs as well.

Delays
A delay tells LoadTest how long to pause before it runs the next item in the schedule. You can insert a delay into a schedule or a script. The advantages of inserting it into a schedule are that the delay is visible in the schedule and the delay is easy to change without editing the script.

In performance testing, you use delays to model user behavior. For example, if your Accounting user group calculates the hours and taxes, and then pauses for two minutes, you would add a delay after the script containing the tax calculation.

Synchronization Points
A synchronization point lets you coordinate the activities of a number of users by pausing the execution of each user at a particular point (the synchronization point) until one of the following events occurs:

All users associated with the synchronization point arrive at the point.
A timeout period is reached before all users arrive at the point.
You manually release the users while monitoring the schedule.

When one user encounters the synchronization point, the user stops and waits for the other users to arrive. When the set number of users reach the synchronization point, LoadTest releases the users and allows them to continue executing.

Transactors
A transactor tells LoadTest the number of items each user will run in a given time period. For example, you might be testing an Order Entry group that completes 10 forms per hour. Or you might be testing a Web server, and you want the server to be able to support 100 hits per minute. To model this time-based behavior, you use a transactor.

The procedures for adding scenarios, executables, delays, synchronization points, and transactors to a schedule are found in Chapter 7 of the Using Rational LoadTest manual, which can be found on the documentation CD.

4. Set the order in which the schedule items will run.

A selector provides more sophisticated control than running a simple sequence of consecutive items in a schedule. A selector tells LoadTest which items each user will execute, and in what sequence. For example, you may want to use a selector to repeatedly select a script at random from a group of scripts.

LoadTest provides the following types of selectors:

Sequential ù Runs each script or scenario in the order in which it appears in the schedule. This is the default.

Next Available ù Distributes its scripts or scenarios to an available user. This is typically used in a GUI user group. The items are parceled out in order, based on which users are available to run another script. Once an item runs, it does not run again.

Random with Replacement ù The selector runs the items under it in random order, and each time an item is selected, the odds of it being selected again remain the same.

Random without Replacement ù The selector runs the items under it in random order, but each time an item is selected, the odds of it being selected again change.

Dynamic Load Balancing ù With dynamic load balancing, items are not selected randomly. This means the next script is selected to balance the load according to the weight that you have set. You can balance the load either for time or frequency.

The procedure for inserting selectors into a schedule is found in Chapter 7 of the Using Rational LoadTest manual, which can be found on the documentation CD.

Rational Unified Process