Beowulf Parallel Linux

Objective: Evaluate feasibility of achieving Gips-scale workstation performance with ensembles of commodity personal computer (PC) subsystems in support of Earth and space science requirements.

Approach: Demonstrate a high-performance and high performance-to-cost multiple computer workstation providing high disk I/O bandwidth and disk capacity in support of Earth and space science application demands. Integrate 16 PC subsystems, 16 high-capacity, low-cost disk drives; multiple Ethernets; and high-resolution video monitors and drivers. Port the Linux Unix-like operating system to processing nodes. Add necessary software upgrades to Linux to facilitate and improve support for parallel and distribute computing. Promote and support application development on the workstation. Test performance of various applications, algorithms, and benchmarks on the workstation.

Accomplishments: The Beowulf Demonstration system, Beowulf-2, was completed on schedule. As planned, Beowulf-2 incorporates advances in commodity technology, including 16 Intel Pentium processors (100 MHz clock), 512 Mbytes of memory, 20 Gbytes of disk storage, dual 100 Mbps Fast Ethernet internal networks, and two high-resolution monitors. The Linux operating system has been augmented with advanced channel-bonding software to support efficient, user-transparent, multiple parallel Ethernet network channels. Important systems software advances include ports of the MPI message passing protocol and the Condor job stream scheduling subsystems. Extensive studies of the Beowulf-1 parallel communications and file transfer performance characteristics have been conducted and the findings published at professional conferences.

Significance: In the spirit of cheaper/better/faster, this project demonstrates the potential of exploiting very inexpensive and widely available components for high-performance computing. The Beowulf approach examines the viability of achieving scalable internal communication by means of multiple parallel commodity networks. It moves the disk I/O operating point for users of large datasets from remote file servers over LAN's to local n-way parallel direct disk to memory access. Beowulf provides a vehicle for technology transfer as a testbed for evaluating experimental distributed systems software and through Linux distribution channels, disseminating the results of Beowulf software development to a broad user community.

Status/Plans: With the completion of the Beowulf prototype and demonstration units, near-term activities focus on evaluation, system software, and application development, with the objective of determining the effectiveness of the new price-performance operating point established by the Beowulf approach in satisfying NASA mission requirements. Among the experiments planned is an advanced internal network topology, which may deliver many times the current sustainable communications bandwidth. The Grendel distributed operating environment will be developed to assist in the user management of the ensemble of resources. The University of Maryland Jovian-2 parallel disk access software package will be ported to provide high-bandwidth file transfer.

Graph: Beowulf parallel workstation demonstrates scaling of commodity networks for balanced mass storage management.

Point of Contact:

Dr. John E. Dorband
Goddard Space Flight Center
dorband@nibbles.gsfc.nasa.gov
301-286-9419