1. Inicio keyboard_arrow_right
  2. Job Deployment

Job Deployment

Job deployment in Geryon2 is handled by the TORQUE resource manager, which is based on the original PBS. The Maui Cluster scheduler is integrated to TORQUE to improve the manageability and efficiency of the scheduler. Some of the basic commands available are listed in the table below

  • showq: Display all running jobs
  • pbsnodes: List the status and attributes of all nodes in the cluster
  • qsub: Submit a job
  • qstat: Show job status
  • qdel: Delete a job

Job submission can be done in one of two ways: either call qsub from the command line, passing all the necessary flags, or prepare a submission script. A typical BASH script would be like this:

#!/bin/bash
#PBS -V
#PBS -N XXXnameXXX
#PBS -k eo
#PBS -q XXXqueueXXX
#PBS -l nodes=XXXmpinodesXXX:ppn=XXXmpicoresXXX
#PBS -l walltime=hh:mm:ss
#################################

### Switch to the working directory;
cd $PBS_O_WORKDIR

### Run:
load module openmpi-1.10.4
mpirun -np XXXnprocessXXX XXXcommandXXX
echo "done"

In the example, replace XXXnprocessXXX with the number of processes to run and XXXcommandXXX with the name of your program. To submit this job, simply type


$> qsub ./example.sh

For more mpirun options, type mpirun --help on a terminal. The meanings of the PBS flags passed in the script are:

  • -V : Inherit all the current environmental variables
  • -N XXXnameXXX : Set the job name as `XXXnameXXX'
  • -k eo : Keep output and error from the job
  • -q XXXqueueXXX : Specify the queue `XXXqueueXXX'. Currently jobs are sent to default queue, so this line can be safely omitted from the script(only required for newton queue users)
  • -l nodes=XXXmpinodesXXX:ppn=XXXmpicoresXXX : Specify the number of nodes and cores per node for your job
  • -l walltime=hh:mm:ss : Inform PBS the anticipated run-time for your job (e.g. for a~1~hour job, use walltime=01:00:00)

We encourage users to look into the documentation of Maui to learn about its functionality and how to take advantage of it.

Interactive mode for Testing

It is possible to launch an interactive console using qsub -I -l nodes=1:ppn=1 where you can open a new terminal inside the runing job where you can do initial testing of your script or compile. It is not recommended for larger runs or compute something longer than a cople minutes since it grabs the terminal and any interruption/disconnect will kill the job.

Code Libraries

Geryon2 makes use of the Environment Modules package. This provides a dynamic modification of a user's environment via modulefiles.

Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command, which interprets the modulefiles.

Typically modulefiles instruct the module command to set or modify shell environment variables such as PATH, MANPATH, LD_LIBRARY_PATH, INCLUDE_DIR, etc. Modulefiles may be shared by many users on a system, and users may have their own collection to supplement or replace the shared modulefiles.

Modules can be loaded and unloaded dynamically and atomically, in a clean fashion. All popular shells are supported, including bash, ksh, zsh, sh, csh and tcsh, as well as some scripting languages such as Perl and Python.

Modules are useful in managing different versions of applications. Modules can also be bundled into metamodules that will load an entire suite of different
applications.

To check the list of available modules in Geryon2, type

 $> module avail 
----------------------------- /usr/share/Modules/modulefiles------------------------------------------
atlas              fftw2_gnu_float    fftw3_gnu          gsl_gnu            hdf5-parallel      module-git         null               
dot                fftw2_intel        fftw3_intel        gsl_intel          hdf5-serial        module-info        openmpi-1.10.4
fftw2_gnu          fftw2_intel_float  gcc-4.9.2          hdf5-1.10.5-serial intel              modules            python-2.7.6
use.own

-------------------------------- /etc/-----------------------------------------------------------------

Examples:

  • To load the gnu GSL library for your job, include the following line in your script:

module load gsl-gnu

  • To load the Intel suite of compilers, use this:

module load intel

Modules can also be loaded from the command line. Modules are active for the duration of a session, or until they are explicitly unloaded with the module unload command. For more details check the man pages
for the module command.

For python users who require special libraries or python>3.6.8, we recomend install miniconda in their HOME dir.