Although today’s handy laptops perform many advanced and computationally intensive tasks, projects involving Big Data require significantly more resources. That need is satisfied by the HPC infrastructure, built from a network of computing clusters combined with immense memory. Access to these resources is remote, so job submission and data preview occurs through an interface on any local computing machine from any (allowed) geolocation. The HPC infrastructure is a shared community space, so you might want to familiarize yourself with the usage policy to avoid disrupting peer work.
Table of Contents
1. Introduction to HPC infrastructure1.1 ACCESS - an advanced computing and data resource program
1.1.1 XSEDE supercell storage
1.2 SCINet Scientific Computing: HPC, high-speed networking and training
1.2.1 SCINet: Atlas computing cluster
1.2.2 SCINet: Ceres computing cluster
1.2.3 SCINet: Juno storage
1.3 ISU HPC Network
1.3.1 ISU HPC: Condo computing cluster
1.3.2 ISU HPC: Nova computing cluster
1.3.3 ISU HPC: LSS system
2. Remote Access to HPC resources
2.1 Virtual Private Network (VPN) Connection (command line)
2.2 Secure Shell Connection (SSH) (command line)
2.2.1 SSH shortcuts and password-less login
2.3 Open On Demand (OOD) Connection (web-based GUI)
3. Setting up your home directory for data analysis
3.1 Example .bashrc file configuration
4. Software Available on HPC
4.1 Accessing pre-installed modules
4.2 Accessing software via package manager
4.3 Installing custom programs in user space
5. Introduction to job scheduling
5.1 SLURM: basics of workload manager
5.1.1 Introduction to SLURM
5.1.2 SLURM commands
5.1.3 Creating SLURM job submission scripts
5.1.4 Submitting dependency jobs using SLURM
5.2 PBS: Portable Batch System
5.2.1 PBS commands
5.2.2 Creating PBS job submission scripts
5.2.3 Submitting dependency jobs using PBS
6. Introduction to GNU parallel
7. Introduction to containers
7.1 Apptainer: the container system for secure HPC (singularity rebranding)
7.1.1 Singularity (archival)
Creating containers using Singularity
Modifying existing containers
Singularity on your Mac via Vagrant
7.2 Docker - accelerated container application development