Performance - Measuring System Performance


The measurement of system performance is a complex and arduous task. Many different components can affect the outcome of a performance run and the performance results produced. The engineers within Amdahl's performance lab take care to identify these components and ensure that the impact associated with each is understood and accounted for in the performance measurement.

System Used for Following Results

The performance work done on the A+Edition was focused on improving the large system performance characteristics of Solaris when running heavy commercial workloads. While improving large server performance, we did not want to degrade the performance of the smaller servers. The preliminary lab results presented in this section were gathered on a SPARCcenter 2000 system. The system included up to 16 50-MHz SuperSPARC CPUs and 2 gigabytes of main memory.

Benchmark Workloads

A large number of benchmarks exist in the industry, each measuring a particular aspect of a system. Some concentrate on integer or floating point performance, while others focus on transaction processing, or "typical" UNIX command usage, or network server capability. Unfortunately, any particular benchmark may not give a good indication of "real-world" performance at a particular customer site if the benchmark does not mimic that customer's workload.

The Amdahl performance lab used a number of industry standard benchmarks to characterize the performance of the A+Edition. The results indicate that the A+Edition provides substantial performance improvements for the SPEC SDM benchmarks, which simulate heavy multiuser workloads typically found in commercial environments. A+Edition does not, however, provide significant performance improvements for other common benchmarks, such as Laddis, TPC-A, and TPC-B. This is because these benchmarks are already highly tuned by Sun.

Customers that have environments that are similar to the SPEC SDM workload can expect similar performance improvements. These workloads are characterized by a large number of processors, concurrent users and memory, varied transactions, and a relatively large proportion of system time or operating system usage.

Preliminary results for SPEC SDM SDET and KENBUS are included in this white paper. Note that these results are not yet "audited", that is, they are not on file with SPEC at this time, but they have been included here to give a preliminary view of the performance gains available on the A+Edition.

Improvements in Base Solaris 2.4 Performance

When performance measurements are generated in the Amdahl performance lab, tests are run against both a base Solaris 2.4 system as well as the A+Edition Rel 1.1 system. Each of these systems are tuned to provide the best performance for the given benchmark. In running the baseline tests using the Solaris 2.4 system provided by Sun, it was immediately apparent that significant improvements in system performance had been made by Sun.

These performance improvements were especially noticeable in larger system configurations and thus SPEC SDET performance for base Solaris 2.4 was significantly higher then that provided in base Solaris 2.3. Improvement in SPEC KENBUS performance was also noted. Since many of the performance enhancements provided by the A+Edition are complementary to the changes made in base Solaris, the performance of Release 1.1 of the A+Edition has also been dramatically improved over Release 1.0 which was based on Solaris 2.3. The results contained in the remainder of this section compares the A+Edition Release 1.1 to base Solaris 2.4 as well as to the A+Edition Release 1.0.