First Research Projects Chosen for NAS CRAY J90

by Chuck Niggley

Nine projects have been selected for work to be performed on the CRAY J90 testbed system, newton. These projects, encompassing 14 applications, were selected to represent the first phase of newton users -- in keeping with the original plan to put a small number of projects (fewer than 10) on the system during the first year. Users began accessing the system in July.

A total of 17 proposals -- all noteworthy -- were received in response to a Request for Proposal released last May to current NAS CRAY C90 users and other selected customers. It is anticipated that other projects will be added to the system in early 1997, following some scheduled hardware upgrades and enhancements to the support software systems, including MPI (Message Passing Interface). Although some of the researchers who submitted proposals already had well-tuned parallel applications using MPI, it was decided that the system's performance was not adequate to support those users at present.

Much Preparation for New Users

Several important events have taken place since the newton system was announced (see the March-April issue of NAS News for background and configuration). The four systems were installed in late February and passed acceptance tests. New and updated software packages were installed, and various systems configurations were tested for I/O performance and ease of use for users. During this time, a version of the PBS scheduler was written for newton and tested. Cray Research Inc.'s beta version of the MPI software was also installed and tested. Various NPB (NAS Parallel Benchmarks) were ported and timing runs accomplished. In preparation for the new users, documentation was published on the World Wide Web (wwW).

Below are summaries of the nine projects selected, listing NASA locations or company names, and Principal Investigators (PIs).

Boeing Commercial Airplane Group

Jeffrey Lewis, PI. The NPARC Navier-Stokes code is a general purpose CFD (computational fluid dynamic) tool which is applicable to a wide variety of aerospace design and analysis problems involving fluid flow. It is actively supported by the NPARC Alliance, a partnership between NASA Lewis Research Center and the Arnold Engineering Development Center.

TLNS3D solves three-dimensional, time-dependent, thin-layer Navier-Stokes equations with a finite-volume formulation on structured grids. Expertise will be gained in applying and developing load-balancing tools to convert grid block topology that is defined by the geometric configuration to a grid block topology that will make efficient use of a desired number of processors.

Lockheed Martin Corp.

Erich Bender, PI. FALCON is a structured-grid, multiblock, finite-volume Navier-Stokes code for general-purpose analysis of aerodynamic and propulsion flow fields.

SPLITFLOW is an upwind, finite-volume, unstructured-grid CFD code with automatic grid generation and adaptation for the Euler analysis of flow over complex geometries. The purpose of this project will be to test the performance of these vectorized codes in a large-scale parallel environment and to improve the parallel efficiency and scalability of these codes on systems of this class.

McDonnell Douglas Aerospace

Samson Cheung, PI. CFL3D solves the three-dimensional, time-dependent, thin-layer Navier-Stokes equations with a finite-volume formulation on structured grids. Work on this code will be to implement a solution-restart capability. The overset grid and patched grid methodologies need to be tested. It also needs a three-level multigrid capability; at present, it has two levels. The benchmark results must match those of the serial counterpart.

Northrop Grumman Corp.

Kari Appa, PI. ENSAERO is a NASA-developed Euler and Navier-Stokes CFD computer program that is capable of handling multiblock grid models and flexible wing-body combinations. Aerodynamic analysis incorporating multiple maneuvers in a single computational run has never before been accomplished and, when implemented, will reduce the time required for an aircraft CFD analysis by hundreds of hours. The code must be optimized for multinode efficiency.

NASA Ames Research Center

Karen Gundy-Burlet, PI. At present, aircraft engine compressors are among the least efficient elements of an aircraft engine. This is due to the complex, unsteady, three-dimensional nature of the flow, coupled with the adverse pressure gradient inherent in compressors. A three-dimensional, unsteady, thin-layer Euler/Navier-Stokes zonal code (STAGE-3) has been developed to analyze these flows. This code will be implemented using the High Performance Fortran (HPF) compiler.

NASA Ames Research Center

Dennis Jespersen, PI. The OVERFLOW computer code is a widely used Navier-Stokes solver. The code handles complex geometries by allowing multiple zones with arbitrary overlapping, interpolating from one zone to another to provide appropriate boundary conditions. Splitting zones across nodes implies that the implicit solvers in OVERFLOW would have to be augmented by linear solvers that can span nodes.

NASA Goddard Space Flight Center

Jose Zero, PI. The proposed experiments are part of an international effort (organized under the World Climate Research Programme's Climate Variability and Predictability project) to assess predictability on seasonal time scales. Improved understanding and prediction of seasonal-to-interannual variability is a national priority of the U.S. Global Change Research Program and has recently been designated as a high priority for NASA's Mission to Planet Earth project.

The Data Assimilation Office (DAO) runs a full General Circulation Model (GCM) of the terrestrial atmosphere as part of its Data Assimilation System. Large parts of the GCM have been ported to a message-passing paradigm and are currently being tested in several High Performance Computing and Communications Program facilities.

The fluid dynamics module DYCORE is functional but its scaling characteristics are prone to degradation under latencies above 100 microseconds. DAO proposes to work in conjunction with the NAS staff to reduce existing high latencies between nodes and at the same time work toward a more robust DYCORE code that will depend less on network latencies.

NASA Langley Research Center (LaRC)

Christopher Riley, PI. LAURA (Langley Aerothermodynamic Upwind Relaxation Algorithm) has been the workhorse analysis tool for LaRC's Aerothermodynamics Branch for the past several years. At present, an elementary MPI version of the code exists and has been run on the IBM SP2 at LaRC.

The DPLUR (Data Parallel Lower Upper Relaxation) code was developed under a grant at the University of Minnesota. It was created specifically for parallel architectures and was developed on a Thinking Machines CM-5. The intent is for this code to eventually have the same functionality as the LAURA code.

FELISA (Finite Element Langley Imperial Swansea Ames) is a closely coupled, unstructured grid generator/flow solver. Although the flow solver codes function in a parallel environment, they have not been optimized for performance.

NASA Langley Research Center

Veer Vatsa, PI. The TNLS3D code will combine microtasking with message passing to attain a high level of scalability, where coarse-grained parallelization (across blocks) will be achieved across the nodes with message passing, and finer grained parallelization (within blocks) will be achieved through microtasking.

Users Get Individualized Help

The NAS scientific consulting group is implementing a new support model for assisting newton users in porting and tuning their codes. User groups assigned to newton have a specific consultant designated to be their interface to all other NAS staff (such as the systems group), as necessary.

Planned enhancements to the newton cluster over the next few months include adding Cray Research Inc.'s GigaRing hardware to support faster communications among the four systems (Q1FY97). Several performance-enhanced MPI releases are expected during this period. In addition, the HSP and parallel systems staff are working on performance improvements to the system and developing utilities to help users; for example, the release of P2D2, the parallel debugger developed by the NAS applications and tools group.

More information on newton, including CRAY J90 documentation.