Objective: To develop parallel software for neural network implementation on Multiple Instruction Multiple Data (MIMD) machines, based on decomposition optimization technology.
Approach:
Accomplishments: Several neuron partitioning strategies for neural networks have been proposed: partitioning nodes (or neurons) with patterns on the output layer of the neural network, partitioning layers with patterns, and partitioning nodes with patterns.Two versions of software corresponding to each partitioning strategy have been developed and examined on simulated data of virtual reality image on CM-5 machines. Evaluations were made as well.Two conference papers were written. One was accepted for presentation at the International Conference on Parallel and Distributed Processing Techniques and Applications, 11/95, Athens, GA. The second paper was submitted to the World Congress of International Federation of Automatic Control, to be held in 6/96 in San Francisco, CA, and is under review.
Significance: The efficiency of newly proposed parallel algorithms for training neural networks has been improved significantly. This improvement has been verified through software testing. Different strategies for partitioning the neural networks present different processing granularity aimed at various parallel computers. Some strategies are suited for tight-coupled parallel computers and others for loose-coupled parallel computers. Specifically, the strategy of partitioning nodes with patterns on the output layer (which has the properties of regular communication pattern, lower communication overhead, and higher granularity) shows very promising computational performance and flexibility on MIMD machines. All software is developed in the user-friendly manner. Although the software is developed on CM-5 machine, it can be ported to other parallel computers with simple modifications.
Status/Plans: The software testing on the CM-5 is on hold, due to the current status of the CM-5. Work on porting the software to the Intel Paragon is underway now. Porting the software to other MIMD platforms, such as the IBM SP-2, is being performed and will continue through the end of this project.
Point of Contact:
Dr. Xin Feng
Department of Electrical and Computer Engineering
Marquette University
fengx@vms.csd.mu.edu
414-288-3504