Rui's Blog
Comment on page

Lecture 1: Course Overview

Course Description

This grad-level course seeks to:
  1. 1.
    Provide an overview of various advanced computing software and hardware solutions
  2. 2.
    Introduce CUDA for parallel computing on the Graphics Processing Unit (GPU)
  3. 3.
    Introduce the OpenMP solution to enabling parallelism across multiple CPU cores
  4. 4.
    Introduce the Message Passing Interface (MPI) standard for leveraging parallelism on a CPU cluster
  5. 5.
    Promote an understanding instrumental in deciding what parallel computing model is suitable for which problems.

Linux "module" utility

Linux module usage
[dan@euler ~]$ gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
[dan@euler ~]$ module load gcc/6.4.0
[dan@euler ~]$ gcc --version
gcc (GCC) 6.4.0
[dan@euler ~]$ nvcc -o cudaprogram
bash: nvcc: command not found
[dan@euler ~]$ module avail cuda
--------------/usr/local/share/modulefiles ---------------------
cuda/0_user/cuda cuda/7.5 cuda/8-rc cuda/9 cuda/9.1
cuda/7 cuda/8 cuda/8.0 cuda/9.0
[dan@euler ~]$ module load cuda/9
[dan@euler ~]$ nvcc -o cudaprogram
[dan@euler ~]$ module list
Currently Loaded Modulefiles:
1) gcc/6.4.0 2) gcc/0_cuda/6.4.0 3) cuda/9
[dan@euler ~]$ module unload cuda gcc
[dan@euler ~]$ nvcc
bash: nvcc: command not found

The Euler cluster

Slurm (Simple Linux Utility for Resource Management)

Slurm is used on Euler for job management and scheduling.
Slurm usage (SBATCH flags documentation) can be found here.
Example of a Slurm-specific batch script
#!/usr/bin/env bash # intepret file as bash script
#SBATCH --job-name=HelloScript
#SBATCH-p wacc # a partition is a logical chunk of cluster
#SBATCH --time=0-00:00:10
#SBATCH --output=“hello_output-%j.txt”
#SBATCH --ntasks=1 --cpus-per-task=1 # simple jobs: one core suffices
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=8 # for mpi
#SBATCH --cpus-per-task=4 # multithreaded jobs
#SBATCH --gres=gpu:1 # gres: Generic RESource
# --gres=type[:model]:N
# e.g., gpu:gtx1080:3or infiniband:1
#SBATCH --constraint=haswell
# regular bash script
cd $SLURM_SUBMIT_DIR # directory where the script is submitted from
echo “Hello, $name_str!
sbatch (Slurm batch) usage
[dan@euler~]$ sbatch # submit a scheduling script to Slurm
Submitted batch job 1975385
[dan@euler~]$ cat hello_output-1975385.txt
Hello, World!