Table of Contents

01. Introduction to Data Science

• Data Science Principles
• Applications of Computational Science
  • Computational Biology
    • go to Bioinformatics Workbook
  • Geospatial Analytics
    • go to Geospatial Workbook
  • Machine Learning Outlook
    • go to AI Workbook
      
      

02. Introduction to the Command Line

• Terminal: a text-based interface for command-line operations
• Introduction to UNIX Shell: shell variables, HOME dir, .bashrc
  • Basic Commands: Navigation, File Creation & Preview
  • Text Files Editors
  • System Info and Access Permissions
  • Admin Commands
  • Tutorial: Getting Started with UNIX + VIDEO
• Useful Text Manipulation Programs
  • Tutorial: GREP – simple search for regular expressions
  • Tutorial: SED – edit stream text
  • Tutorial: AWK – advanced text processing
  • Tutorial: BIOAWK – biological data manipulation
• UNIX CheatSheet
  
  

03. Setting Up Computing Machine

• Operating System Installation
  • Windows OS Installation
  • Linux OS Installation
• Must-Have Software
  • Basic Office Software
  • Basic Developer Tools
  • Basic Developer Libraries
• Various Methods of Software Installation
  • Tutorial: Installations on MacBook Pro
  • Tutorial: Installations on Windows
  • Tutorial: Installations on Linux
    
    

04. Development Environment

• Integrated Development Environment (IDE)
  • Tutorial: Visual Studio Code
• Python Programming Environment
  • Jupyter: Interactive Web-Based Multi-Kernel Programming Interface
    • Tutorial: Getting Started with JupyterLab
    • Tutorial: Getting Started with Jupyter Notebook
    • Tutorial: Sharing Jupyter Notebooks via MyBinder
  • PyCharm: IDE for Professional Python Developers
• R Programming Environment
  • RStudio: Integrated Environment for R Programming
    • Tutorial: Setting Up RStudio
      
      

05. Introduction to Programming

• Basics of Algorithm Structure
• Introduction to Bash Scripting
• Introduction to Python programming
  • Tutorial: Python Setup
• Introduction to R programming
  
  

06. High-Performance Computing (HPC)

• Setting up your home directory for data analysis
  • .bashrc example file
• Introduction to HPC infrastructure
  • XSEDE Supercomputer
    • XSEDE Supercell Storage
  • SCINet Network
    • Atlas Computing Cluster
    • Ceres Computing Cluster
    • Juno Storage
  • ISU HPC
    • Condo Computing Cluster
    • Nova Computing Cluster
    • LSS Storage
• Secure Shell Connection (SSH)
  • SSH Shortcuts
  • Password-less SSH
• Remote Data Access
  (see more in section 7: Data Acquisition: Remote Data Access ⤴)
• Software Available on HPC
  • Accessing pre-Installed Modules
  • Accessing software via package manager
  • Installing Custom Programs in User Space
• Introduction to Job Scheduling
  • SLURM
    • Introduction to SLURM
    • SLURM Cheatsheet
    • Tutorial: Creating SLURM Job Submission Scripts
    • Tutorial: Submitting Dependency Jobs using SLURM
  • PBS
    • PBS Cheatsheet
    • Tutorial: Creating PBS Job Submission Scripts
    • Tutorial: Submitting Dependency Jobs using PBS
• Introduction to GNU Parallel
• Introduction to Containers
  • Singularity
    • Tutorial: Creating Containers using Singularity
    • Tutorial: Modifying Existing Containers
    • Tutorial: Singularity on your Mac via Vagrant
  • Docker
    
    

07. Data Acquisition and Wrangling

• Remote Data Access
  • Remote Data Transfer
    • Tutorial: Copying Data using SSH
    • Tutorial: Copying Data using Globus
    • Tutorial: File Transfer using irods
    • Tutorial: File Transfer using SRA Toolkit
    • Tutorial: Downloading Online Data using WGET
    • Tutorial: Downloading Online Data using Web Scraping
    • Tutorial: Downloading Online GitHub Repos using GIT
    • Tutorial: Downloading Online GitHub Folders using SVN
  • Remote Data Preview without Downloading
    • Tutorial: Viewing Text Files using UNIX commands
    • Tutorial: Viewing PDF Files using X11 SSH connection
    • Tutorial: Mounting Remote Folder on Local Machine
• Data Manipulation
  • Manipulating Excel Data Sheets
    • Create Workbook from Multiple Text Files
    • Export Multiple Worksheets as Separate Text Files
    • Create Index for All Worksheets
    • Merge Two Spreadsheets using a Common Column
  • Manipulating Text Files with Python
    • Tutorial: Read, Write, Split, Select Data
    • Tutorial: JSON Module - Encoding & Decoding JSON Data
    • Tutorial: Math Module - Various Mathematical Functions
    • Tutorial: Pandas Library - Data Structure Manipulation Tool
    • Tutorial: Numpy Library - Multi-Dimensional Arrays Parser
    • Tutorial: SciPy Library - Algorithms for Scientific Computing
• Data Wrangling: ready-made apps
  • Merge files by common column (python)
  • Aggregate data over slicing variations (python)
    
    

08. Data Visualization

• Introduction to Scientific Graphic Design
  • Raster Graphics Tools
  • Vector Graphics Tools
  • Adobe Creative Cloud
  • Template-based Web Tools
• Introduction to Scientific Graphing
  • Gnuplot: Creating Plots in the UNIX Shell
  • Plotly-Dash: Data Processing & Interactive Plotting with Python
    • Introduction to Plotly (Python library)
    • Introduction to Dash (Python library)
    • Interactive Graphing – Local Server with Web-Based Interface
    • Plotly Graphing - Interactive Examples in the JupyterLab
      • Tutorial: Creating XY Scatter Plot
      • Tutorial: Creating 1D Volcano Plot
      • Tutorial: Creating Heatmap
      • Tutorial: Creating Dendrogram
      • Tutorial: Creating Clustergram (Heatmap with Dendrograms)
  • RStudio: Data Processing & Plotting with R
    • Setting Up an RStudio Environment
    • Tutorial: Creating Boxplots in R
    • Tutorial: Creating Heatmaps in R
    • Tutorial: Creating Heatmaps in R using ComplexHeatmap
      
      

09. Project Management

• Introduction to Project Management

• Project Management Tools
  • Introduction to GitHub
    • Tutorial: Git commands (part 1)
    • Tutorial: Git commands (part 2)
  • Introduction to BitBucket
  • Introduction to ZenHub
  • Tutorial: Research Reproducibility
• Documentation Improvement Tools
  • Introduction to Markdown
• Team Communication Tools
  • Introduction to Slack