The ROCK Linux MPP Project

Project Description

The Massively Parallel Processing(MPP) project was initiated to provide a stable, heterogeneous parallel processing platform on top of a Rock Linux cluster*. The core design of the project depends heavily on the stability and availability of the Rock Linux ports. Current work is centered on the main Intel x86 based builds, though building on the ports should not be too difficult.

Software

Current software available for the MPP project includes MPICH, LAM, and PVM message passing libraries. MPICH and LAM are independent implementations of the MPI message passing specification. The MPI specification is the newest platform for message passing and is gaining popularity despite the lack of standardization of some important functionality. MPICH has evolved as the portability standard and so most message passing packages are designed to handle its implementation. LAM is a valid very portable MPI that provides similar calls to MPICH and dynamic scheduling. PVM - Parallel Virtual Machine is the original portable message passing library, it has a wide selection of packages available. We also provide scripts to run mulitple instances of programs across the cluster. These can be used to run programs that have isolated data sets and do not need interprocess communication such as Distributed.net or SETI@Home.

BSP might also now be a possibility.

Additional Software
Rock Linux MPP also includes a large set of parallel computational library packages including ScaLAPACK, P_ARPACK, and FFTW. It also provides several parallel development systems including Cactus Code and HPF. Data visualizers such as OpenDX and Vis5d+ are available and allow for the display of data in large formats including HDF5 and netCDF. On going work includes WAN remote queueing and abstraction systems, NetSolve and Globus. Of course, we will also provide pre-built MPICH and PVM based POVRAY packages as well.

File Systems
Currently, we have been working on just vanilla NFS. We have tested MFS/DFSA for Mosix as well as GFS for 2.4.3. We hope to also test PVFS with ROMIO

Intel x86
The Intel message passing systems have been complimented by the inclusion of the Mosix extensions to Linux. Mosix allows for peer-to-peer dynamic scheduling of processes. While using a Mosix enabled kernel, message passing processes may be dynamically scheduled for optimization inside the cluster. Mosix also includes its own distributed file system MFS that has a good deal to offer Mosix based clusters. Mosix also provides a decent set of process accounting tools.Mosix is currently testing on 2.4.3 kernel.

Misc
Rock Linux will soon include SciLab for MatL*b compatible code with support for ScaLAPACK calls over PVM. SciLab should allow a great deal of library integration for both single node libraries through PVM and parallel libraries. Octave is also available for single node processing.

Cactus code, LCA, and Flexio work currently from CVS. Until the Rock Package Evolution project integrates CVS these will be not included by default. They work great and are quite impressive otherwise.

Compiler
All computational packages are set to default compile with g77. g77 is far from optimum and it is recommended that all serious installations purchase a commercial Linux FORTRAN compiler, especially for work in F90 and F95. We hope to provide a facility for mass optimized recompilation of the computational packages using the Rock Linux build system at a later date. This is highly dependent on the efforts of the Rock Package Evolution as well.

Status

Packages
Most packages work. Mosix will be release once 2.4.3 issues are solved, similar for GFS. BSP and most of the rendering packages are not finished. SciLab does not compile on Rock, but 2.6 binaries do work which is promising.

Queuing and Process Accounting
We do not have a completely satisfactory solution at this time for all of the available messaging systems. We recommend using queue for a GPL system. OpenPBS is free to download after registering and will soon be released under a BSD license. All work is being done favoring OpenPBS.

Mosix
Mosix has some funkiness to its 2.4.3 kernels, mostly the kernels fault. This may be resolved really soon.

Future
Scyld has given us verbal permission to integrate their technologies into Rock Linux. When a 2.4.x version of bproc is available we shall use it. Also, we will integrate as many of their installation technologies into the entirety of Rock Linux

Papers, Notes, Howtos, etc.

Documentation for installation will not be done until Rock Linux has a usable installation system. We should be working on that and in the process integrate clustering setup.

Also, we really need help testing and debugging, so if you have two x86 systems, a good grasp of math, and can code in FORTRAN...

I have written a small paper so that I could plug myself and Rock Linux at Singapore Linux World Conference 2001. It is here

*NOTE

Parallel Processing, in this definition, is the scientific/computational cluster that uses the multiple processor and memory resources to solve complex or large formulae. High Availability is a type of cluster that provides a Quality of Service solution, usually involving multiple processing elements and redundant networking elements tied to a redundant memory element to eliminate downtime and increase service throughput. Clusters may also be generally used to serve large numbers of requests or store large amounts of data among the cluster nodes. All these clusters do, however, provide solutions that are orthogonal in implementation and need. So it would be possible to design a cluster that is redundant, holds large data resources, and solves large mathematical formulae. In fact, many large installations of scientific clusters do use all these systems. Rock Linux is currently planning to provide HA support at a later date.

MPP Developers

Chris Hamilton (US/SG)

All there is. Sorry to disappoint.