NASA
High Performance Computing
and Communications Program

Computational AeroSciences Project

Multithreaded System for Distributed Memory Environments

Objective: To design and implement a distributed memory multithreaded environment which is portable across massively parallel architectures and heterogeneous networks of workstations.

Approach: Lightweight threads are becoming increasingly useful in supporting parallelism and asynchronous control structures in applications and language implementations. Traditionally, lightweight threads are supported only within the single address space of a process, or in shared memory environments with multiple processes. Likewise, interprocess communication systems do not currently allow messages to be sent directly entities within a process. We have been building a a runtime system, Called Chant supporting lightweight threads in a distributed memory environment called Chant, to remedy these problems. Chant combines standard interfaces for lightweight threads, pthreads, and interprocess communication, MPI, to support point-to-point communication between any two threads in a distributed memory system. The design is divided into four layers: point-to-point communication, remote service requests, remote thread operations, and collective communication for thread groups.

Accomplishments: Chant provides support for ropes, a group of threads capable of executing data parallel code. In particular, ropes provide support for relative indexing of threads and collective communication among the threads comprising the rope. These two features allow message passing programs to be executed using a rope with fairly trivial changes to the code. As expected, the ropes layer adds some overhead to the point-to-point communication (see attached figure), however, this allows us to execute multiple data-parallel programs independently within the same set of distributed processes. We have also been experimenting with load balancing using migrating threads. The issue here is to migrate threads in the presence of pointers. We have designed a system which allows threads which use pointers to be migrated within a homogeneous system. With the help of the programmer, the system keeps track of all pointers (both in stack and heap) and updates them on the destination processor. We are in the process of implementing such a system.

Significance: Despite their popularity and utility in shared memory systems, lightweight thread packages designed for distributed memory systems have received little attention. This is unfortunate: in a distributed memory system, lightweight threads can support task-level parallelism; they can overlap communication with computation; they can emulate virtual processors; and they can permit dynamic scheduling and load balancing. However, there is no widely accepted implementation of a distributed memory threads package. Chant fills this need by combining established standards for lightweight threads and interprocess\ communication to create a portable distributed memory threads package.

Status/Plans: We plan to continue optimizing and enhancing the underlying thread system. We also plan to implement the thread-based migration providing other features which allow users to easily balance the load in a distributed computation.

Contact:

Piyush Mehrotra

ICASE, M.S. 403

NASA Langley Research Center

(757)-864-2188

pm@icase.edu

Collaborators:

David Cronk, College of William and Mary

Matthew Haines, University of Wyoming