home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Usenet 1994 October
/
usenetsourcesnewsgroupsinfomagicoctober1994disk2.iso
/
misc
/
volume9
/
tbench
/
README
< prev
Wrap
Text File
|
1989-11-12
|
24KB
|
516 lines
####################################################
# This software released to the public domain #
# without restrictions of any kind. #
####################################################
@(#)README 3.4 11/12/89
The programs included in this, the first release of TBENCH
were all developed by DigiBoard Incorporated (St. Louis Pk, MN)
for internal performance testing. DigiBoard used these to
test their own, and competitors multiport communication
boards. Some of those results have been published in trade
magazines.
These benchmarks are a another attempt at developing objective
benchmarks to compare the performance of tty subsystems under
extreme load. An earlier benchmark suite developed by ANVIL
(Australia) was posted to comp.sys.i386 some time ago. Those
were a fine first attempt, but did suffer from some limitations.
At DigiBoard, we believe these benchmarks provide a much
better indication of steady state tty I/O performance.
In particular they send data for a length of time before
benchmarking begins, and after it completes, so that the
results are closer to indicating steady-state performance.
They also check input data automatically, and provide a
summary of the errors detected.
DigiBoard has made an attempt to keep these benchmarks as
objective as possible. However DigiBoard is a for-profit
company, and as such must have included some bias. Anyone
using these benchmarks is heartily encouraged to analyze
them, improve them, and if they see fit, to criticize them
as needed.
These programs have been tested on essentially all UNIX
ports to 286/386 PC-compatible systems, and also on Suns.
Send bug fixes, enhancements, comments (and flames) to:
Gene H. Olson uunet!digibd!gene
Member of Technical Staff
DigiBoard Inc.
St. Louis Park, MN (612) 922-8055
------------------------------------------------------------
The remainder of this file describes how to use the terminal
benchmark programs. We assume the reader is an experienced
UNIX user, thoroughly familiar with the concept of tty devices
and modes, and very comfortable at the UNIX command level.
Novice UNIX users may find these notes incomprehensible.
THE PROGRAMS
------------
Before you can use these benchmarks, you need to compile two
programs, and run one of them to calibrate it for use on your
system. Enter the following commands:
cc -o perf -O perf.c
cc -o cpu cpu.c
./cpu -c
When compiling perf.c and cpu.c you are wise to make sure your
system is using setvbuf() with the correct parameter sequence.
The original UNIX 5.2 release had parameters 2 and 3 reversed
from the setbuf(3) manual page, and from subsequent releases of
UNIX. However while the manual pages were different, the lint(1)
library file was consistent with the way the parameters were
actually used. I suggest you run lint on these programs before
attempting to use them. If you get an error on the setvbuf()
declaration, then use instead:
cc -DSETVBUF -o perf -O perf.c
cc -DSETVBUF -o cpu cpu.c
./cpu -c
The programs now available include:
perf C program which may be invoked for either input or
output testing.
In output mode, the program writes 6 digit numbers
(plus checksum) to standard output, which should be
redirected to the tty under test. The program reports
the output speed, real (elapsed) time, and system
time consumed.
In input mode, perf reads the output of another perf
in output mode, redirected through a pair of tty ports.
The program compares the actual data received to the
expected input and reports error statistics. The
report includes the speed of input, real time, system
time, the number of codes missing or in error, and
the number of stray characters received.
cpu C program to determine how busy the system is during
input/output testing.
This program must first be run on an idle system to
calibrate its internal loop. Thereafter it may be
invoked to drop its priority and absorb all available
idle time. Using a combination of reported system time
and its own internally calibrated loop, it displays
the amount of CPU time consumed elsewhere at both
process and interrupt levels.
The assignment of time to process and interrupt levels
is very approximate, but the total is an accurate
reflection of the CPU being utilized by other programs.
If the only other programs running are the terminal
benchmarks, the results are quite good.
If your system contains memory of different speeds,
this program may report grossly inaccurate results.
There is no solution, except to change memory boards
or move to another system.
obench A shell file to supervise output testing.
The program takes as parameters a list of ttys,
test modes and baud rates. It runs the perf program
on 1 to n of these terminals in all of the mode and
baud rate combinations given. The results of these
tests are written to standard output, and should be
captured to a file.
oprint A shell file to summarize the results reported by
the obench shell file above.
islave A shell file to monitor input terminals and report
the speed and accuracy of the input actually received.
This program takes as parameters a baud rate,
and a list of terminals to be monitored. The
data coming into the terminals may include test
commands and data supplied by the ibench program
below. The results are written to standard output,
and should be captured to a file.
ibench A shell file to supervise input testing and to produce
test data for by the islave program above.
Ibench takes as parameters a list of ttys, test modes,
and baud rates. It runs the perf program to write
test commands and data to the given ttys. These
ttys are then connected to terminals monitored by the
islave program above.
The commands sent out by ibench cause the islave program
to print messages, change input modes and baud rates
as the test progresses.
iprint A shell file to summarize the results reported by
the islave shell file above.
OUTPUT TEST PROCEDURE
---------------------
Output testing is straightforward, and requires minimal hardware
setup.
The benchmark writes data out to multiple ports as fast as possible,
without flow control, so it is not necessary to hookup the cables to
terminals.
To perform the test on a DigiBoard PC/8e under SCO Xenix, type
the following command on the system console:
obench 9600 exta extb opost -opost /dev/ttyi1[a-h] | tee junk
This command outputs data at baud rates 9600, exta (19200),
and extb (38400) to the given ports in both opost (cooked) and
-opost (raw) modes. All combinations of baud rates and modes
are tested, using 1-8 terminals each. This yields a total of 3 baud
rates * 2 modes * 8 terminals = 48 different tests, each writing
about 100K characters to 1 or more terminals.
If all goes well, the test takes about 2 hours. The test output
is displayed on the terminal and written to the file junk. The
output should look similar to:
TEST tty=1, baud=9600, mode=opost
TIME process=5.7, interrupt=-0.1, total=5.6
/dev/ttyi1a : OUT cps=965, real=103.58, user=0.4, sys=5.1
TEST tty=2, baud=9600, mode=opost
TIME process=11.3, interrupt=-0.1, total=11.2
/dev/ttyi1b : OUT cps=965, real=103.54, user=0.5, sys=5.1
/dev/ttyi1a : OUT cps=965, real=103.54, user=0.4, sys=4.9
TEST tty=3, baud=9600, mode=opost
TIME process=16.1, interrupt=0.5, total=16.6
/dev/ttyi1b : OUT cps=965, real=103.58, user=0.4, sys=5.0
/dev/ttyi1c : OUT cps=965, real=103.56, user=0.5, sys=5.2
/dev/ttyi1a : OUT cps=965, real=103.56, user=0.3, sys=5.0
TEST tty=4, baud=9600, mode=opost
TIME process=21.7, interrupt=0.7, total=22.3
/dev/ttyi1b : OUT cps=965, real=103.56, user=0.4, sys=5.1
/dev/ttyi1c : OUT cps=965, real=103.56, user=0.6, sys=5.0
/dev/ttyi1d : OUT cps=965, real=103.62, user=0.4, sys=5.0
/dev/ttyi1a : OUT cps=965, real=103.62, user=0.4, sys=5.0
TEST tty=5, baud=9600, mode=opost
TIME process=28.3, interrupt=-0.3, total=27.9
/dev/ttyi1b : OUT cps=965, real=103.60, user=0.6, sys=5.1
/dev/ttyi1d : OUT cps=965, real=103.56, user=0.4, sys=5.0
/dev/ttyi1e : OUT cps=965, real=103.56, user=0.6, sys=5.1
/dev/ttyi1a : OUT cps=965, real=103.60, user=0.5, sys=5.1
/dev/ttyi1c : OUT cps=963, real=103.82, user=0.6, sys=4.8
TEST tty=6, baud=9600, mode=opost
TIME process=32.4, interrupt=1.0, total=33.5
/dev/ttyi1c : OUT cps=964, real=103.64, user=0.5, sys=4.8
/dev/ttyi1f : OUT cps=965, real=103.56, user=0.6, sys=5.1
/dev/ttyi1a : OUT cps=965, real=103.60, user=0.4, sys=5.0
/dev/ttyi1e : OUT cps=965, real=103.56, user=0.4, sys=5.2
/dev/ttyi1b : OUT cps=965, real=103.56, user=0.5, sys=4.8
/dev/ttyi1d : OUT cps=965, real=103.56, user=0.6, sys=4.9
TEST tty=7, baud=9600, mode=opost
TIME process=36.8, interrupt=3.1, total=39.9
/dev/ttyi1d : OUT cps=965, real=103.62, user=0.4, sys=5.1
/dev/ttyi1f : OUT cps=964, real=103.70, user=0.4, sys=5.1
/dev/ttyi1b : OUT cps=957, real=104.48, user=0.5, sys=4.9
/dev/ttyi1g : OUT cps=964, real=103.72, user=0.4, sys=4.9
/dev/ttyi1c : OUT cps=962, real=103.94, user=0.4, sys=5.0
/dev/ttyi1a : OUT cps=964, real=103.64, user=0.5, sys=4.9
/dev/ttyi1e : OUT cps=958, real=104.34, user=0.7, sys=5.0
TEST tty=8, baud=9600, mode=opost
TIME process=45.0, interrupt=0.5, total=45.5
/dev/ttyi1a : OUT cps=961, real=103.96, user=0.6, sys=4.9
/dev/ttyi1d : OUT cps=963, real=103.78, user=0.6, sys=5.1
/dev/ttyi1f : OUT cps=964, real=103.66, user=0.5, sys=5.1
/dev/ttyi1c : OUT cps=963, real=103.74, user=0.6, sys=5.1
/dev/ttyi1h : OUT cps=957, real=104.46, user=0.5, sys=5.1
/dev/ttyi1b : OUT cps=955, real=104.70, user=0.4, sys=4.8
/dev/ttyi1g : OUT cps=963, real=103.80, user=0.4, sys=5.1
/dev/ttyi1e : OUT cps=964, real=103.68, user=0.6, sys=5.0
TEST tty=1, baud=9600, mode=-opost
TIME process=1.6, interrupt=0.0, total=1.6
/dev/ttyi1a : OUT cps=960, real=104.16, user=0.5, sys=1.1
TEST tty=2, baud=9600, mode=-opost
TIME process=2.9, interrupt=0.2, total=3.1
/dev/ttyi1b : OUT cps=959, real=104.18, user=0.5, sys=0.9
/dev/ttyi1a : OUT cps=960, real=104.14, user=0.4, sys=0.9
TEST tty=3, baud=9600, mode=-opost
...
When the test is complete, you may summarize the data in tabular
form by typing:
oprint junk
This prints a summary of the input received in tabular format with
column headers. The output should resemble:
ports baud mode cps real %sys %host
----- ------ ------ ----- ------ ----- ----
1 9600 opost 965.0 103.6 5.1 5.31
2 9600 opost 965.0 103.5 10.2 5.31
3 9600 opost 965.0 103.6 15.2 5.24
4 9600 opost 965.0 103.6 20.4 5.28
5 9600 opost 964.6 103.6 25.5 5.29
6 9600 opost 964.8 103.6 30.6 5.29
7 9600 opost 962.0 103.9 36.6 5.43
8 9600 opost 961.3 104.0 41.7 5.42
1 9600 -opost 960.0 104.2 1.1 1.17
2 9600 -opost 959.5 104.2 2.2 1.12
3 9600 -opost 959.0 104.2 3.2 1.11
4 9600 -opost 959.5 104.2 4.3 1.12
5 9600 -opost 958.8 104.2 5.3 1.11
6 9600 -opost 959.7 104.2 6.4 1.10
7 9600 -opost 958.6 104.2 8.7 1.29
8 9600 -opost 958.6 104.2 9.7 1.27
1 19200 opost 1919.0 104.2 10.2 5.29
2 19200 opost 1922.0 104.0 20.4 5.31
3 19200 opost 1921.7 104.0 30.5 5.28
4 19200 opost 1920.8 104.1 40.6 5.29
5 19200 opost 1920.4 104.1 50.8 5.29
6 19200 opost 1920.2 104.1 61.0 5.30
7 19200 opost 1919.6 104.2 71.5 5.32
8 19200 opost 1917.4 104.3 81.3 5.30
1 19200 -opost 1921.0 104.1 2.1 1.07
...
The columns printed are:
baud The test baud rate.
mode The communication mode, either opost (cooked) or -opost
(raw).
flow The flow control mode, either ixon (output flow control)
-ixon (no flow control).
cps The character per second (cps) output speed actually measured.
Under ideal conditions, this will be the baud rate divided
by 10. Lower numbers indicate a reduction in performance.
%sys Percent of system time used during the steady-state portion of
the test. This is computed by starting with 100%, subtracting
the "perf" program user time, and the idle time measured by
the "cpu" program. Because it is a subtractive method, it
is less accurate at lower loads, and progressively more
accurate as the load increases.
%host Percent of system time used per kilo-character of output.
INPUT TEST PROCEDURE
--------------------
Input testing is much harder than output testing. To do a good job,
you need two systems, the system being tested, and another system
to produce the test data. You must cable the two systems together,
and run programs on both systems. In all, it is about 3 times as
much work as the output test procedure.
For reference, call the system producing the test data the "producer",
and the system under test the "consumer". If you wish to test
input on 8 lines simultaneously, you must cable 8 ports on the
producer to 8 ports on the consumer system. You will probably need
a gender changer and a null modem connector for each line as well.
Most intelligent cards will do output much faster than they do input,
so generally the consumer system will overload before the producer
system begins to slow down. This is not always true however, so
you should make sure your results do not suffer because the producer
system cannot run full speed.
I suggest you check out the lines with "cu" before you start testing
to make sure everything really works. I can tell you from experience
it is frustrating to discover a bad line when the input test is
3/4 complete.
With a DigiBoard PC/8e under SCO Xenix, begin the test by typing the
following command on the console of the consumer system:
islave 9600 /dev/ttyi1[a-h] | tee junk
Then go to the console of the producer system and type:
ibench 9600 exta extb ixoff -icanon /dev/ttyi1[a-h]
The producer system will begin sending commands and test data to
the consumer system; the consumer system will report test results
on the console and in the file "junk".
Since the producer system sends commands as well as test data to
the consumer system, it is essential that communication between the
two systems remains intact during the test. Remember that you may
be seriously overloading the input capacity of the consumer system,
causing test commands to be lost. If this phenomenon occurs, it
may be necessary to run the tests by hand.
You should check the output on the consoles of both the producer
and consumer systems as the test progresses. The same number of
display lines are written to both terminals in a very similar format,
so it is easy to see if there is a problem.
The output written to the consumer display, and captured in the
file junk should be very similar to the following:
TEST tty=1, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.0, interrupt=6.5, total=6.5
/dev/ttyi1a : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=2, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.0, interrupt=12.4, total=12.4
/dev/ttyi1e : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=3, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.0, interrupt=16.7, total=16.8
/dev/ttyi1b : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=959, real=104.20, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1e : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=4, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.0, interrupt=21.7, total=21.7
/dev/ttyi1e : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1b : IN cps=959, real=104.18, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1f : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=959, real=104.18, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=5, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.1, interrupt=26.5, total=26.6
/dev/ttyi1a : IN cps=959, real=104.18, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1f : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1e : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1b : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1c : IN cps=959, real=104.18, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=6, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.0, interrupt=31.2, total=31.2
/dev/ttyi1e : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1b : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1f : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1c : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1g : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=7, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.1, interrupt=36.1, total=36.1
/dev/ttyi1b : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1f : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1e : IN cps=959, real=104.18, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1c : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1g : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1d : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=8, baud=9600, mode=-icanon, flow=ixoff
TIME process=0.2, interrupt=40.1, total=40.3
/dev/ttyi1b : IN cps=956, real=104.52, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=956, real=104.52, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1f : IN cps=956, real=104.56, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1e : IN cps=956, real=104.52, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1d : IN cps=956, real=104.52, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1c : IN cps=956, real=104.52, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1g : IN cps=956, real=104.60, user=0.0, sys=0.1, errs=0, stray=0
/dev/ttyi1h : IN cps=956, real=104.60, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=1, baud=9600, mode=icanon, flow=ixoff
TIME process=0.0, interrupt=13.4, total=13.4
/dev/ttyi1a : IN cps=960, real=104.14, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=2, baud=9600, mode=icanon, flow=ixoff
TIME process=0.0, interrupt=24.4, total=24.5
/dev/ttyi1e : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
/dev/ttyi1a : IN cps=960, real=104.16, user=0.0, sys=0.0, errs=0, stray=0
TEST tty=3, baud=9600, mode=icanon, flow=ixoff
TIME process=12.6, interrupt=22.6, total=35.2
...
Note that both the "errs" and "stray" counts should ideally be zero
during this test until the consumer system becomes overloaded.
Each system is different, but it is usually easy to see when things
are beginning to go wrong.
When the test is complete, you should then run iprint to format
the input results you got above. Type the following command on
the consumer system:
iprint junk
This action will produce a report similar to the following:
ports baud mode flow cps errs %sys %host
----- ------ ------ ------ ----- ------ ------- ------
1 19200 -icanon ixoff 1920.0 0.0 8.51 4.43
2 19200 -icanon ixoff 1913.0 0.0 14.03 3.67
3 19200 -icanon ixoff 1919.0 0.0 20.44 3.55
4 19200 -icanon ixoff 1919.3 0.0 26.65 3.47
5 19200 -icanon ixoff 1919.8 0.0 33.17 3.46
6 19200 -icanon ixoff 1919.5 0.0 38.88 3.38
7 19200 -icanon ixoff 1919.4 0.0 44.79 3.33
8 19200 -icanon ixoff 1919.1 0.0 49.69 3.37
9 19200 -icanon ixoff 1919.6 0.0 57.42 3.32
10 19200 -icanon ixoff 1919.2 0.0 63.54 3.31
11 19200 -icanon ixoff 1919.6 0.0 69.85 3.31
12 19200 -icanon ixoff 1919.0 0.0 75.96 3.30
13 19200 -icanon ixoff 1919.5 0.0 81.57 3.27
14 19200 -icanon ixoff 1918.3 0.0 83.60 3.11
15 19200 -icanon ixoff 1872.4 0.0 83.60 2.98
16 19200 -icanon ixoff 1828.6 0.0 83.24 2.84
1 38400 -icanon ixoff 3839.0 0.0 12.03 3.13
2 38400 -icanon ixoff 3840.0 0.0 21.45 2.79
3 38400 -icanon ixoff 3839.7 0.0 31.28 2.72
4 38400 -icanon ixoff 3838.8 0.0 41.11 2.68
5 38400 -icanon ixoff 3839.6 0.0 50.43 2.63
...
The columns printed are:
baud The test baud rate.
mode The communication mode, either icanon (cooked) or -icanon
(raw).
flow The flow control mode, either ixoff (input flow control)
-ixoff (no flow control).
cps The character per second (cps) input speed actually measured.
Under ideal conditions, this will be the baud rate divided
by 10. Lower numbers indicate a reduction in performance.
Higher numbers indicate the consumer system clock is running
slow during the test; check with a stopwatch.
errors Number of 6 digit + checksum codes missing from the input
stream. When this number goes non-zero, you may assume you
have begun to overload the producer system.
%sys Percent of system time used during the steady-state portion of
the test. This is computed by starting with 100%, subtracting
the "perf" program user time, and the idle time measured by
the "cpu" program. Because it is a subtractive method, it
is less accurate at lower loads, and progressively more
accurate as the load increases.
%host Percent of system time used per kilo-character of input
data received.
If there are any obvious inconsistencies in the input, iprint will
insert an error line to let you know. You may wish to just edit out
the input data in error, and replace it with corresponding comment
lines explaining the problem.
If you wish, you may edit comment lines into your raw data files
to identify details of the test, and problems you encountered.
Such comments lines must contain a pound sign (#) in column 1.
Comments are passed transparently through both oprint and iprint.