The Beowulf Scalable Mass Storage research project will implement a trivially reproducible system that provides at least a terabyte of secondary storage and a gigabyte/s aggregate external bandwidth. The performance characteristics of the system will be evaluated on real applications in the operational environment provided by the Earth and Space Sciences (ESS) Project.
The Beowulf Scalable Mass Storage research project builds on the Beowulf Parallel Linux technology and methodology. The availability of the Linux operating system and the performance levels attained by PC-market components, particularly disks and networks, has created the opportunity to provide cost-effective solutions to secondary storage problems. It is now feasible to construct a high-performance system entirely of commodity-off-the-shelf (COTS) components. Using an enhanced version of the Linux operating system provides a vendor independent solution that promotes future robustness and cost-effectiveness. The availability of source and the strong network support inherent in Linux is important to the mass storage system. We require the ability to modify and enhance the kernel functionally to construct a system that will serve bulk data from component nodes that were designed for a single-user, multitasking environment. In addition, using Linux allows us to easily harvest the software built for networks of workstations.
Among the software projects being considered at this time are:
The remaining key issues in this research and development effort are the driving applications and technology transfer. The system is being developed within the infrastructure established to support the Grand Challenge Investigators for the second phase of the ESS Project. This will allow us to test the research system in an operational environment with a large, heavily used parallel computing system at its core. The Beowulf Scalable Mass Storage project has been identified by the Scalable I/O Initiative as a target testbed. This direct involvement with a large number of diverse institutions either requiring or developing high-performance I/O systems will provide independent performance evaluation and serve as one of the vehicles for technology transfer. The primary vehicle will be Linux itself; any enhancements to the operating system or application software will be made available on the World Wide Web through the usual Linux distribution mechanisms.
The skeletal system has been assembled. It contains a full complement of nodes but is only partially populated with disks and interconnection networks. Specifically, it is composed of 60 Pentium Pro CPU's, 2.5 gigabytes of aggregate core memory, and 300 gigabytes of disk space. The system is connected by 100 megabit/s Ethernet. Total cost in July 1997 was only $102,000 .
A critical technology for this project is Linux support for COTS networks. Linux has always provided support for the multitude of vendors that produce Ethernet hardware. To satisfy the high-bandwidth requirements for this project, we have become involved with producers of high-performance networks. We have integrated a four-port Myrinet switch into the skeletal cluster and have completed initial tests to verify Myricom's performance claims. We have also become a beta test-site for Gigabit Ethernet produced by Packet Engines, Inc. The first high-performance driver for Gigabit Ethernet was for Linux and was developed by this project. Our involvement with innovative networking companies ensures that Linux, and hence the Beowulf Mass Storage project, will maintain its advantage in networking performance.
The performance/cost ratio for available COTS disk technology continues to increase at a significant rate. We have an ongoing program to evaluate these disks as they become available. Current results motivated the purchase of 40 6.4 gigabyte IDE drives for the skeletal system.
The Parallel Virtual File System from Clemson University has been ported to the Beowulf cluster. This system is based on PVFS servers that run on the nodes of the cluster and a set of application programmer libraries. A number of out-of-core computations have been implemented on Beowulf, and their performance is being evaluated.
Even the modest capabilities of the skeletal system (compared to the fully populated system) will provide enough performance to attract applications from the Scalable I/O Initiative and the ESS Grand Challenge Investigators. Establishing an application base in the second phase of the development will set the stage for an effective technology transition in the third phase and will help tune the system as its capabilities increase.
The primary focus for the next phase of development will be associated with the interconnection networks. By continuing to work with all the vendors that produce high-speed networks for PC-class hardware, we plan to maintain the Beowulf's performance advantage within the cluster computing domain. In particular, we plan to continue to develop Linux drivers for any technology that will bridge the gap between our Beowulf cluster and the high-performance Silicon Graphics/Cray platforms that form the ESS testbed. The parallel file systems used on this project will operate on a variant of a client-server model. In order to exploit the high-speed network within the cluster, the functionality of the IOP daemons (the servers) must be split to match its hierarchical structure. We plan to design and implement these server daemons and begin evaluating them on the skeletal system. The client side of the parallel file system is being developed on Beowulf clusters. The next step is to integrate this software into the ESS testbed environment and provide a user interface that fits the programming model for the CRAY T3E.
Dr. Phillip R. Merkey
Center of Excellence in Space Data and Information Sciences (CESDIS)
Goddard Space Flight Center
merk@cesdis.gsfc.nasa.gov
301-286-3805
http://cesdis.gsfc.nasa.gov/linux-web/beowulf/beowulf.htm
Table of Contents | Section Contents -- Testbeds