NASA
High Performance Computing
and Communications Program
Computational AeroSciences Project
MPI-IO
Objective:
The goal of the MPI-IO interface is to provide a widely used standard for describing parallel I/O operations within an MPI message-passing application. The interface should establish a flexible, portable, and efficient standard for describing independent and collective file I/O operations by processes in a parallel application. The MPI-IO interface is intended to be submitted as a proposal for an extension of the MPI standard in support of parallel file I/O.
Approach:
Working together, IBM Research and NASA Ames have drafted MPI-IO, a proposal to address the portable parallel I/O problem. In a nutshell, this proposal is based on the idea that I/O can be modeled as message passing: writing to a file is like sending a message, and reading from a file is like receiving a message. MPI-IO intends to leverage the relatively wide acceptance of the MPI interface in order to create a similar I/O interface. MPI-IO supports a high-level interface to describe the partitioning of file data among processes, a collective interface describing complete transfers of global data structures between process memories and files, asynchronous I/O operations, and optimization of physical file layout on storage devices.
Accomplishments:
At an April meeting of the MPI Forum, members decided to add an I/O chapter to the MPI-2 standard. At the Forum's June meeting, NAS staff presented MPI-IO as a starting point for this effort. The Scalable I/O Initiative has already adopted MPI-IO as its mid-level API in May, and will work with the MPI Forum to create a common parallel I/O interface standard.
MPI-IO is an extension to the popular MPI (Message Passing Interface) standard. Together, they will enable truly portable programs to be written for parallel machines. NAS has implemented a generic portable version of MPI-IO, called PMPIO. This prototype achieved a 20-fold speedup in writing files on the IBM SP2. To date, PMPIO has been ported to the IBM SP2, Silicon Graphics Inc. (SGI) and Sun Microsystems shared-memory workstations, an Intel Paragon, and the NAS CRAY J90. This portability allows the group to measure the performance characteristics of MPI-IO on vastly different architectures.
Significance:
Thanks to MPI, writing portable message passing parallel programs is almost a reality. One of the remaining problems is file I/O. Although parallel file systems support similar interfaces, the lack of a standard makes developing a truly portable program impossible. It is not feasible to develop large scientific applications from scratch for each generation of parallel machine, and, in the scientific world, a program is not considered truly portable unless it not only compiles, but also runs efficiently.
Point of Contact:
Leigh Ann Tanner
NASA Ames Research Center
MS 258-6
Moffett Field, CA 94035
tanner@nas.nasa.gov
(415)604-4306