Sharu Paul

Sharu Paul

Sharu Paul is a Research Scientist at the Genome Informatics Facility at Iowa State University. His background is in Biotechnology with a Ph.D. in Genetics and Genomics. His research is wide-ranging, covering topics from Single-Cell Analyses to the building of Deep Neural Networks. By employing an extensive range of bioinformatic analyses, he seeks to solve big data problems and contributes to developing workshops and tutorials for USDA scientists.

DataScience Workbook / 06: High-Performance Computing (HPC) / 1. Introduction to HPC infrastructure / 1.3. ISU HPC Network

HPC Clusters at ISU

High Performance Computing (HPC) refers to a cluster of interconnected computers or servers used to perform complex computational tasks. HPC clusters can process large amounts of data in a short time and can handle large computationally intensive jobs. General information about HPC clusters available at Iowa State University can be found at the ISU HPC Website ⤴.

Components of HPC Clusters at ISU

  • headnode or login node - used to log in, submit jobs, and view output
  • data transfer node - used for data transfers
  • compute nodes - different types including regular, fat, GPU, etc. are used for computations
  • Infiniband switch - connects all nodes

Research Clusters

  • Condo Computing Cluster - Free access cluster
  • Nova Computing Cluster - Faculty can purchase nodes and storage on this cluster

Education Cluster

  • HPC Class - These are partitions on Nova cluster dedicated to class use

LSS Storage

  • Large Scale Storage (LSS) - Storage service for ISU researchers

Logging in

To log in to Nova/Condo Clusters, you need to be connected to ISU network on-campus or use ISU VPN ⤴ if you are off-campus. The first time you try to log in to Nova or Condo, a new Google Authenticator (GA) account is created. You will need to use GA for login.

The first time you use the connection to Nova/Condo, it will respond with:

"host key not found, generate hostkey?(yes/no)"

You need to answer yes to continue.

Condo vs Nova Cluster

Feature Condo Nova
Login node @condo2017.its.iastate.edu @nova.its.iastate.edu
Data Transfer node @condodtn.its.iastate.edu @novadtn.its.iastate.edu
Cores per node 16/20/32/40 36/48/64/96
Home directory space 5 GB 5 GB
Workload Manager SLURM SLURM
Partitions Partitions List ⤴ Partitions List ⤴

Tutorials to get you started on different HPC Clusters are available in this workbook.

…about Software available on ISU HPC Check out the SLURM Basics and for running your jobs on HPC cluster. It is useful to have some basic knowledge about command line before using HPC.
Check this Getting started with UNIX (video + exercises) to learn the basics.

Best Practices: Tips

  • Test your code/job on a small scale before submitting it to the cluster, if possible. This can help save time and resources as submitting a job often takes longer due to queue times and uses more resources. For example, you can use interactive nodes to find and fix errors before submitting a job.
  • Make sure your script is optimized for the cluster such that it uses the available resources efficiently.
  • Use proper documentation and version control such as Git to track changes and ease of reproducibility.
  • Manage your files appropriately on the cluster. Cleaning up unnecessary files and organising data/files is important.

Also check out the Introduction to Project Management and A Practical guide to managing research documentation for additional tips.

Where to find help?

Guides provided by ISU

General guides

Data Transfers

  • For transferring data, use the File Transfers ⤴ guide. Each cluster has its own data transfer node, for example, Nova cluster has novadtn.
  • Use Rclone ⤴ guide to sync data between HPC Clusters and Google Drive.
  • Globus Connect ⤴ online data transfer system. Use Globus Connect Personal to copy data between a cluster and your personal computer.

Additional guides