Job Arrays

Job Array Example

Making a new copy of the script and then submitting each one for every input data file is time consuming. An alternative is to make a job array using the -t option in your submission script. The -t option allows many copies of the same script to be queued all at once. You can use the PBS_ARRAYID to differentiate between the different jobs in the array.

Lets say you want to run 16 blast jobs. Instead of submitting 16 different jobs, you can submit one job, but use the -T parameter and the PBS_ARRAYID variable.

The -T parameter sets the range of the PBS_ARRAYID variable. So setting it to 0-15 will cause the qsub script to call the script 16 times, with an input of 0, 1 , 2 ,3

EXAMPLE QSUB SCRIPT

<span style="color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px; background-color: rgb(248, 248, 248);">#!/bin/bash</span>
<span style="color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px; background-color: rgb(248, 248, 248);"># ----------------QSUB Parameters----------------- #</span>
<span id="qsub_params_span" style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px;"><span id="bash_content" style="box-sizing: border-box;">#PBS -S /bin/bash</span>
<span id="q_param_input" style="box-sizing: border-box;">#PBS -q default</span>
<span id="l_param_input" style="box-sizing: border-box;">#PBS -l nodes=1:ppn=2,mem=1000mb</span>
<span id="M_param_input" style="box-sizing: border-box;">#PBS -M youremail@illinois.edu</span>
<span id="m_param_input" style="box-sizing: border-box;">#PBS -m abe</span>
<span id="N_param_input" style="box-sizing: border-box;">#PBS -N array_of_blast_jobs</span></span>
<span style="color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px; background-color: rgb(248, 248, 248);"># ----------------Load Modules-------------------- #</span>
<span id="modules_span" style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px;">module load blast/2.2.26</span>
<span style="color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px; background-color: rgb(248, 248, 248);"># ----------------Your Commands------------------- #</span>
<span id="software_span" style="box-sizing: border-box; color: rgb(51, 51, 51); font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 13px; line-height: 26px;">perl run_blast_job.pl $PBS_ARRAYID</span>