Submitting MPI-parallel jobs
1. General information
There are three queues reserved to submit your MPI parallel jobs and they are described here below
| Queue Name | Min. CPU # | Max. CPU # | Max. walltime | Quick submission how-to |
|---|---|---|---|---|
| zebra | 8 | 128 | 12h | qsub -q zebra -l nodes=#nodes:ppn=8 |
| blade | 4 | 256 | 12h | qsub -q blade -l nodes=#nodes:ppn=4 |
| iblade | 4 | 256 | 12h | qsub -q iblade -l nodes=#nodes:ppn=4 |
2. Basic MPI How-To
To run your MPI parallel jobs you need to require some nodes on one of the three queues above and use all of the processors available on each node. To do that include in your PBS script something like
#PBS -q blade #PBS -l nodes=2:ppn=4
which for example requests two blade nodes with 4 processors each.
To run a code that was previously compiled with MPI libraries don't forget to load the appropriate modules you compiled the code with. So for example if you used OpenMPI 1.3 and PGI compilers you should include the following lines in your PBS script file.
source /etc/profile.d/modules.sh module purge module load openmpi/1.3/pgi
Finally to run your job you should use the mpirun command. If you use the suggested OpenMPI the syntax is very simple: you have to simply add a line similar to this below to your PBS script.
mpirun -x PATH -x LD_LIBRARY_PATH ./myexe