# Lecture 28: MPI Parallel Programming: MPI Collectives. Overview of topics covered in the class.

## Lecture Summary

* Last time
  * Wrap up p2p communication
  * Collective communication: Synchronizations, communications, operations
* Today
  * Collective communication: Operations, data types
  * Talk about the final exam. [Past exam](https://uwmadison.app.box.com/s/vkwsrao6og5ocyno7u491n03kybhzdyn)

## MPI Derived Types (did not have time to cover in class)

* Previously we sent/received a contiguous buffer of identical elements of predefined data types
* Now we want to send non-homogenous elements (structure) or chunks that are not contiguous in memory
* MPI Datatypes
  * Primitive datatypes: MPI\_CHAR, MPI\_FLOAT, MPI\_INTEGER, etc.
  * Derived datatypes: Can be constructed by four methods (contiguous, vector, indexed, struct)
* Typemaps
  * Used to describe an MPI derived type
  * Specifies a sequence of primitive data types, and a sequence of integers that represent the byte displacements, measured from the beginning of the buffer
    * Typemap = {(type0, disp0), ..., (typen, dispn)}
* Extent: distance, in bytes, from beginning to end of type
  * E.g., {(double,0),(char,8)} has extent 16
* Type signature: The sequence of primitive data types
  * E.g., for a data type with typemap {(double,0), (int,8), (char, 12)}, its signature is {double, int, char}

![MPI type-definition functions (constructors)](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MXy7nypQh3fne2ZG_3-%2F-MXyB0muOSakJ8W4WwTL%2FScreen%20Shot%202021-04-10%20at%206.48.44%20PM.png?alt=media\&token=cf00b6c9-3ee7-4ec3-b013-34e97c725bbb)

## Takeaways from ME 759

* Know your hardware
* Moving data around is expensive in energy and time
* Seek solution approaches that expose concurrency/parallelism in your problem
* Use the tools of the trade to work like a pro and then get paid like a pro
  * Debuggers, profilers, CMake, memory checkers, compiler flags, git, etc.
  * Do HPC

![](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MXy7nypQh3fne2ZG_3-%2F-MXyOPIxK2-lcxw-gVVt%2FScreen%20Shot%202021-04-10%20at%207.47.13%20PM.png?alt=media\&token=5a291029-874f-4acc-a6ea-61aa6ca91826)

![](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MXy7nypQh3fne2ZG_3-%2F-MXyORs4sjCPh60Duv7e%2FScreen%20Shot%202021-04-10%20at%207.47.23%20PM.png?alt=media\&token=6b5f7e03-c926-4351-9030-ff365fbfb415)

This has been such a fun ride.