HPC Clusters at ISU
High Performance Computing (HPC) refers to a cluster of interconnected computers or servers used to perform complex computational tasks. HPC clusters can process large amounts of data in a short time and can handle large computationally intensive jobs. General information about HPC clusters available at Iowa State University can be found at the ISU HPC Website ⤴.
Components of HPC Clusters at ISU
- headnode or login node - used to log in, submit jobs, and view output
- data transfer node - used for data transfers
- compute nodes - different types including regular, fat, GPU, etc. are used for computations
- Infiniband switch - connects all nodes
Research Clusters
- Condo Computing Cluster - Free access cluster
- Nova Computing Cluster - Faculty can purchase nodes and storage on this cluster
Education Cluster
- HPC Class - These are partitions on Nova cluster dedicated to class use
- Check the Guide to HPC Class ⤴ from ISU for more details
LSS Storage
- Large Scale Storage (LSS) - Storage service for ISU researchers
Logging in
To log in to Nova/Condo Clusters, you need to be connected to ISU network on-campus or use ISU VPN ⤴ if you are off-campus. The first time you try to log in to Nova or Condo, a new Google Authenticator (GA) account is created. You will need to use GA for login.
The first time you use the to Nova/Condo, it will respond with:
"host key not found, generate hostkey?(yes/no)"
You need to answer yes
to continue.
Condo vs Nova Cluster
Feature | Condo | Nova |
---|---|---|
Login node | @condo2017.its.iastate.edu | @nova.its.iastate.edu |
Data Transfer node | @condodtn.its.iastate.edu | @novadtn.its.iastate.edu |
Cores per node | 16/20/32/40 | 36/48/64/96 |
Home directory space | 5 GB | 5 GB |
Workload Manager | SLURM | SLURM |
Partitions | Partitions List ⤴ | Partitions List ⤴ |
Tutorials to get you started on different HPC Clusters are available in this workbook.
…about Software available on ISU HPC
Check this Getting started with UNIX (video + exercises) to learn the basics.
Best Practices: Tips
- Test your code/job on a small scale before submitting it to the cluster, if possible. This can help save time and resources as submitting a job often takes longer due to queue times and uses more resources. For example, you can use interactive nodes to find and fix errors before submitting a job.
- Make sure your script is optimized for the cluster such that it uses the available resources efficiently.
- Use proper documentation and version control such as Git to track changes and ease of reproducibility.
- Manage your files appropriately on the cluster. Cleaning up unnecessary files and organising data/files is important.
Also check out the Introduction to Project Management and A Practical guide to managing research documentation for additional tips.
Where to find help?
- FAQs ⤴ - List of Frequently asked questions such as:
- Contact: hpc-help@iastate.edu
Guides provided by ISU
General guides
- Intro to HPC clusters ⤴
- Intro to Nova cluster ⤴
- Nova OnDemand ⤴
- Virtual Environments ⤴
- Sample Job Scripts ⤴
Data Transfers
- For transferring data, use the File Transfers ⤴ guide. Each cluster has its own data transfer node, for example, Nova cluster has novadtn.
- Use Rclone ⤴ guide to sync data between HPC Clusters and Google Drive.
- Globus Connect ⤴ online data transfer system. Use Globus Connect Personal to copy data between a cluster and your personal computer.
Additional guides
- Unix Introduction ⤴
- SCSLab ⤴
- Matlab Parallel Server ⤴
- ARM DDT Parallel Debugger, MAP profiler and Performance Reports ⤴
- ANSYS RSM ⤴
- Containers ⤴
- Machine Learning Container ⤴
Further Reading
ISU HPC: Condo computing clusterISU HPC: Nova computing cluster
ISU HPC: LSS system
Remote Access to HPC resources
Setting up your home directory for data analysis
Software Available on HPC
Introduction to job scheduling
Introduction to GNU parallel
Introduction to containers
MODULE 07: Data Acquisition and Wrangling