Submitting MPI-parallel jobs

1.  General information

There are three queues reserved to submit your MPI parallel jobs and they are described here below

Queue NameMin. CPU #Max. CPU #Max. walltimeQuick submission how-to
zebra812812hqsub -q zebra -l nodes=#nodes:ppn=8
blade425612hqsub -q blade -l nodes=#nodes:ppn=4
iblade425612hqsub -q iblade -l nodes=#nodes:ppn=4

2.  Basic MPI How-To

To run your MPI parallel jobs you need to require some nodes on one of the three queues above and use all of the processors available on each node. To do that include in your PBS script something like

#PBS -q blade
#PBS -l nodes=2:ppn=4

which for example requests two blade nodes with 4 processors each.

To run a code that was previously compiled with MPI libraries don't forget to load the appropriate modules you compiled the code with. So for example if you used OpenMPI 1.3 and PGI compilers you should include the following lines in your PBS script file.

source /etc/profile.d/modules.sh
module purge
module load openmpi/1.3/pgi

Finally to run your job you should use the mpirun command. If you use the suggested OpenMPI the syntax is very simple: you have to simply add a line similar to this below to your PBS script.

mpirun -x PATH -x LD_LIBRARY_PATH ./myexe

3.  Advanced MPI How-To

3.1  Interactive sessions and command line submission

3.2  Advanced OpenMPI options

3.3  Alternative MPI implementations