DataScience Workbook / 06. High-Performance Computing (HPC) / 5. Introduction to Job Scheduling / 5.1 SLURM: Basics of Workload Manager / 5.1.2 Creating SLURM Job Submission Scripts


Creating SLURM job submission scripts

For a quick list of frequently used SLURM commands/options, take a look at this cheat sheet. This guide will show you how to generate SLURM job submission scripts for your commands, easily and efficiently!

1. Generate commands

We recommend using our run scripts. Run scripts are configured to run frequently used programs with settings that are tailored to fit our needs. The idea is to make it simple to run frequently used programs without having to read the entire manual. Some of the common run scripts are,, runBLASTn etc. Here we will take as an example, but you can use this for almost any of the other run scripts as well as your own commands. First, lets generate the commands:

We have 5 fasta files: file1.fsa, file2.fsa, file3.fsa, file4.fsa and file5.fsa (each with 1000 sequences) for which we need to run BLAST against nr database, we will generate the commands as follows:

for file in file?.fsa; do
echo "./ $file;
done > blastn.cmds

2. Create submission files

The blastn.cmds now contains 5 lines, each set to run blastn on each of the file. To create slurm script for each one of these lines, we run the script as follows: 1 blastn.cmds

Here 1 is to tell that put one job per submission file, and blastn.cmds is the commands file that looks like this:

./ file1.fsa
./ file2.fsa
./ file3.fsa
./ file4.fsa
./ file5.fsa

Once the command is run, you should see 5 .slurm files, each identified by numbers and the command file prefix. The content should look like this:

#SBATCH --ntasks-per-node=16
#SBATCH -t 96:00:00
#SBATCH -J blastn_1
#SBATCH -o blastn_1.o%j
#SBATCH -e blastn_1.e%j
#SBATCH --mail-user=${USER}
#SBATCH --mail-type=begin
#SBATCH --mail-type=end
ulimit -s unlimited
module use /work/GIF/software/modules
./ file1.fsa
scontrol show job $SLURM_JOB_ID

3. Submit the jobs

The next step is to submit the jobs to the cluster. Using the for loop

for f in blastn*.slurm; do
sbatch $f;

All commands will be submitted and will start to run as per the availability of nodes.

Further Reading

Homepage Section Index Previous Next top of page