Scalable Parallel Finite Element Computations of
Rayleigh-Benard-Marangoni Problems
in a Microgravity Environment

Project Director: Graham F. Carey

NASA ESS Cooperative Agreement Number: NCCS5-154

PERIOD COVERED BY THE REPORT: August 16, 1996-August 15, 1997

Objectives:

This research project concerns the analysis and simulation of coupled viscous flow and transport processes including free surface thermocapillary effects. Of particular interest are microgravity applications for manufacturing and life support processes relevant to the U.S. space station, space shuttle program and future space projects. The main thrust of the present research is directed to the development of a highly parallel distributed simulation capability for analyzing 3-D transient and steady state coupled viscous flow and transport processes in this class. However, the simulator will also be more broadly applicable to other problems of interest to NASA, U.S. manufacturing, and science and Engineering. e.g. it can be applied more generally to terrestrial manufacturing processes and will be particularly important to problems involving thin films and coatings since surface tension is also dominant (over gravity) in these applications. Part of the research work also concerns supporting experimental studies on thermocapillary effects in thin films.

Approach:

The approach is based on a Galerkin finite element formulation using hexahedral elements with piecewise continuous triquadratic approximation for the velocity, temperature (or species) fields and trilinear pressure finite element basis. Parallelism is achieved using a domain decomposition of the flow domain and mesh. The resulting discretized problem leads to a sequence of sparse algebraic systems that are solved using a parallel form of generalized conjugate gradient solver. Special attention has been paid to parallel performance issues to meet the performance milestones specified in the contract. The experimental work has been directed to fundamental studies of free surface phenomena for thin films and are being carried out in the Experimental Flows Lab of the Center for Nonlinear Dynamics.

Accomplishments

The performance milestones and schedule for the previous year and the performance levels achieved are listed below:

Milestones

1. Establish project team of programmers, graduate students, post docs, research staff scientists and faculty. Completed update of agreement including negotiated milestones and software deliverables. Exploratory parallel algorithm studies and flow experiments.

2. Cache mirror benchmark tests for computational kernels with scaling analysis to 10 Gigaflops for proposed GSFC testbed. Experimental and theoretical studies. Submitted FY 96 annual report to sponsor via wwW.

3. Achieve at least 10 GigaFlops sustained performance using on the GSFC testbed.

4. Delivered documented version of code and tests to the National HPCC Software Exchange (NHSE). Document cache mirror MVP results. Complete first phase microgravity flow experiments.

5. Parallel Algorithm Research. Test cases and benchmarks. 50 GigaFlop kernel performance achieved on vendor testbed. Submitted FY 97 annual report to sponsor via wwW.

All of the scheduled milestones for this year have been met.

Experimental studies have also progressed well. The basic results have related to observations and analysis concerning long-wavelength instabilities, high spots and interfacial rapture. Further details are provided in the Attachments.

Status and Plans

As indicated above, significant progress has been made on the simulator development and the milestone goals are being met in a timely manner. The first software submission has been recently completed. The next version of the software has been completed in prototype form and is now being tested. We have also achieved the 50 gigaflop and 100 gigaflop performance targets for both kernel performance and for benchmark flow application ahead of schedule. (These performance studies were made by S. Swift at Cray and have been archived for future performance milestone reports.)

Presentations and Publications

The first major publication on the underlying methodology and related work on algorithms was presented in April 1997 in the High Performance Computing Segment of the Annual Meeting of the Society for Digital Simulation. A more specific study presenting the flow formulation, finite element discretization, solution algorithm, parallel aspects and scalability studies has been accepted for Supercomputing '97. Preprints of these papers are included as Attachments I and II. Two papers related to the experimental component of the project and dealing with the long-wavelength instabilities and instability threshold in the Rayleigh-Benard-Maragoni problem are to appear (See Attachment III for more particulars). Two Phd dissertations describing research contributions on the project were also completed during this period.

Papers

Carey, G. F., A. Bose, B. Davis, C. Harle, and R. McLay, Parallel Computation of Viscous Flows, Presented at HPC '97, Atlanta, GA, April, 1997. postscript version

Carey, G. F., C. Harle, R. McLay and S. Swift, MPP Solution of Rayleigh-Benard-Marangoni Flows, to be Presented at Supercomputing '97, San Jose, CA, November 1997. postscript version

Van Hook, Stephen J., M. F. Schatz, J. B. Swift, W. D. McCormick and H. L. Swinney, "Long-wavelength surface-tension-driven Benard convection: experiment and theory," to appear in Journal of Fluid Mechanics.

Van Hook, Stephen J. and J. B. Swift, "Comment II on 'Instability Threshold in the Benard-Maragoni problem'," to appear in Physical Review E.

Theses

Van Hook, Stephen J., "Long-wavelength Instability in Surface-tension-driven Benard Convection," PhD Dissertation, The University of Texas at Austin (November 1996).

Davis, M. B., "Parallel Multilevel Solution of Iteratively Decoupled Transport Problems," PhD Dissertation, The University of Texas at Austin (December 1996).

Symposia/Dissemination

Two special sessions on High Performance Computing and Applications were organized and coordinated by Dr. Carey at the Atlanta HPC '97 Conference to showcase the NASA ESS CAN Teams and disseminate results of their work to the research community. These sessions were well attended and session papers were published in the proceedings volume: High Performance Computing 1997, Grand Challenges in Computer Simulation, Edited by Adrian Tentner, The Society for Computer Simulation, San Diego, CA, 1997. A future workshop on Adaptive Mesh Refinement is being prepared and several participants have been contacted regarding interest and availability. Collaboration/NASA Visits

During the previous year Dr. Carey made visits to NASA JPL and to NASA Goddard to discuss progress on the research, and coordinate/plan research activities. These work trips included discussions on the main parallel algorithm work and applications studies as well as problems in visualizing complex 3D flows. S. Swift, the Cray scientist at Goddard, has been collaborating closely in the research and has consulted regularly and worked with us on performance studies. He has also visited the University to facilitate the interaction. Dr. Carey also visited NCAR in Spring semester and Dr. Spotz from NCAR will spend two weeks in Fall semester collaborating on the project.
To facilitate close interaction, a post-doctoral associate, Dr. B. Davis, has been in residence at NASA Goddard during the year while supported by the U.T. project budget. Dr. A. Bose also spent several months at Cray (Minnesota) working on parallel distributed solver algorithms for the viscous flow problem.

Remarks: Overall, we are very pleased with the progress made on the parallel simulator, its performance and scalability and the fundamental flow experimental work. It should be emphasized that the parallel work is "on the (b) leading edge" in the sense that both the hardware and software for the performance studies is new and frequently not fully stable. That is, hardware failures, compiler bugs and other software bugs are to be expected and will be encountered. We have had to 'weather' some of these difficulties in due course and this has naturally incurred more time and effort than would be the case in a more stable environment.

List of Attachments

I. Paper presented at HPC '97, Atlanta, GA, Parallel Computation of Viscous Flows postscript version
II. Paper to be presented at SC '97, San Jose, CA, MPP Solution of Rayleigh-Benard-Marangoni Flows postscript version
III. Experimental Studies postscript version
IV. Software Submission Report, 'Scalable Parallel Finite Element Computations of Rayleigh-Benard-Marangoni Problems in a Microgravity Environment postscript version
V. Validation Studies postscript version
VI. Software Document postscript version