Changes between Version 4 and Version 5 of WRF4G/ExecutionEnvironments

Timestamp:

Feb 15, 2013 6:04:44 PM (9 years ago)

Author:

carlos

Comment:

--

WRF4G/ExecutionEnvironments

@@ -21 +21 @@
 }}}
 
-= RES =
+= RES - Red Española de Supercomputación =
 
 
 == Altamira ==
+
+{{{
+ssh  <userid>@altamira1.ifca.es
+}}}
 
 === Running Jobs ===
@@ -162 +166 @@
 == !MareNostrum ==
 
+{{{
+ssh  <userid>@mn1.bsc.es
+}}}
+
+=== Running Jobs ===
+LSF is the utility used at MareNostrum III for batch processing support, so all jobs must be run through it. This document provides information for getting started with job execution at the Cluster.
+5.1. Submitting jobs
+A job is the execution unit for LSF. A job is defined by a text file containing a set of directives
+describing the job, and the commands to execute. Please, bear in mind that there is a limit of 3600
+bytes for the size of the text file.
+5.1.1. LSF commands
+These are the basic directives to submit jobs:
+• bsub < job_script
+submits a “job script” to the queue system (see below for job script directives). Remember to pass
+it through STDIN '<'
+• bjobs [-w][-X][-l job_id]
+shows all the submitted jobs.
+• bkill <job_id>
+remove the job from the queue system, canceling the execution of the processes, if they were still
+running.
+5.1.2. Job directives
+A job must contain a series of directives to inform the batch system about the characteristics of the
+job. These directives appear as comments in the job script, with the following syntax:
+#BSUB -option value
+#BSUB -J job_name
+The name of the job.
+#BSUB -q debug
+This queue is only intended for small tests, so there is a limit of 1 job per user, using up to 64 cpus
+(4 nodes), and one hour of wall clock limit.
+#BSUB -W HH:MM
+NOTE: take into account that you can not specify the amount of seconds in LSF. The limit of wall
+clock time. This is a mandatory field and you must set it to a value greater than the real execution
+time for your application and smaller than the time limits granted to the user. Notice that your job
+will be killed after the elapsed period
+#BSUB -cwd pathname
+The working directory of your job (i.e. where the job will run). If not specified, it is the current
+working directory at the time the job was submitted.
+#BSUB -e/-eo file
+The name of the file to collect the stderr output of the job. You can use %J for job_id. -e option will
+APPEND the file, -eo will REPLACE the file.
+7
+MareNosutrm III User's Guide
+#BSUB -o/-oo file
+The name of the file to collect the standard output (stdout) of the job. -o option will APPEND the
+file, -oo will REPLACE the file.
+#BSUB -n number
+The number of processes to start.
+#BSUB -R"span[ptile=number]"
+The number of processes assigned to a node.
+We really encourage you to read the manual of bsub command to find out other specifications that
+will help you to define the job script.
+man bsub
+5.1.3. Examples
+
+Sequential job :
+{{{
+#!sh
+#!/bin/bash
+#BSUB -n 1
+#BSUB -oo output_%J.out
+#BSUB -eo output_%J.err
+#BSUB -J sequential
+#BSUB -W 00:05
+
+./serial.exe
+}}}
+
+The job would be submitted using:
+{{{
+#!sh
+bsub < ptest.cmd
+}}}
+
+Sequential job using OpenMP :
+{{{
+#!sh
+#!/bin/bash
+#BSUB -n 1
+#BSUB -oo output_%J.out
+#BSUB -eo output_%J.err
+#BSUB -J sequential_OpenMP
+#BSUB -W 00:05
+
+export OMP_NUM_THREADS=16
+
+./serial.exe
+}}}
+
+Parallel job :
+{{{
+#!sh
+#!/bin/bash
+#BSUB -n 128
+#BSUB -o output_%J.out
+#BSUB -e output_%J.err
+# In order to launch 128 processes with 16 processes per node:
+#BSUB -R"span[ptile=16]"
+#BSUB -x # Exclusive use
+#BSUB -J parallel
+#BSUB -W 02:00
+# You can choose the parallel environment through modules
+
+module load intel openmpi
+mpirun ./wrf.exe
+}}}
+
+Parallel job using threads:
+{{{
+#!sh
+#!/bin/bash
+# The total number of MPI processes:
+#BSUB -n 128
+#BSUB -oo output_%J.out
+#BSUB -eo output_%J.err
+# It will allocate 4 MPI processes per node:
+#BSUB -R"span[ptile=4]"
+#BSUB -x # Exclusive use
+#BSUB -J hybrid
+#BSUB -W 02:00
+# You can choose the parallel environment through
+# modules
+
+module load intel openmpi
+# 4 MPI processes per node and 16 cpus available
+# (4 threads per MPI process):
+
+export OMP_NUM_THREADS=4
+
+mpirun ./wrf.exe
+}}}
+
 = National Computational Infrastructure (Australia) =
 [http://nf.nci.org.au/facilities/]