Lecture 28: MPI Parallel Programming: MPI Collectives. Overview of topics covered in the class.

Lecture Summary

• Last time

  • Wrap up p2p communication

  • Collective communication: Synchronizations, communications, operations

• Today

  • Collective communication: Operations, data types

  • Talk about the final exam. Past exam

MPI Derived Types (did not have time to cover in class)

• Previously we sent/received a contiguous buffer of identical elements of predefined data types

• Now we want to send non-homogenous elements (structure) or chunks that are not contiguous in memory

• MPI Datatypes

  • Primitive datatypes: MPI_CHAR, MPI_FLOAT, MPI_INTEGER, etc.

  • Derived datatypes: Can be constructed by four methods (contiguous, vector, indexed, struct)

• Typemaps

  • Used to describe an MPI derived type

  • Specifies a sequence of primitive data types, and a sequence of integers that represent the byte displacements, measured from the beginning of the buffer

    • Typemap = {(type0, disp0), ..., (typen, dispn)}

• Extent: distance, in bytes, from beginning to end of type

  • E.g., {(double,0),(char,8)} has extent 16

• Type signature: The sequence of primitive data types

  • E.g., for a data type with typemap {(double,0), (int,8), (char, 12)}, its signature is {double, int, char}

MPI type-definition functions (constructors)

Takeaways from ME 759

• Know your hardware

• Moving data around is expensive in energy and time

• Seek solution approaches that expose concurrency/parallelism in your problem

• Use the tools of the trade to work like a pro and then get paid like a pro

  • Debuggers, profilers, CMake, memory checkers, compiler flags, git, etc.

  • Do HPC

This has been such a fun ride.