> For the complete documentation index, see [llms.txt](https://blog.ruipan.xyz/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://blog.ruipan.xyz/earlier-readings-and-notes/cs759-hpc-course-notes/lecture-8-parallel-computing.-flynns-taxonomy.-amdahls-law..md).

# Lecture 7: Parallel Computing. Flynn's Taxonomy. Amdahl's Law.

## Lecture Summary

* Wrap up Virtual Memory
* Intuitions for Parallel Computing
* Flynn's Taxonomy
* Amdahl's Law

## Why Parallel Computing?

Sequential computing is facing these steep hills to climb:

* Memory Wall: Speed difference between CPU & memory outside the chip
* ILP Wall
* Power Wall: Latency & limited communication bandwidth beyond chip boundaries

### Memory Wall

![](/files/-MT4a2iWQ7l99EZ7X4TY)

Take-home message: Try to stay away from long and winding conversations with the main memory

### ILP Wall

![ILP elicits very complex microarchitecture](/files/-MT4axpClWcbfFSwnde4)

Instruction pipelining; Superscalar execution; Out-of-order execution; Register renaming; Speculative execution; Branch prediction

Predicting the future comes at the cost of microarchitecture complexity and power cost

### Power Wall

Power, and not manufacturing, limits traditional general-purpose microarchitecture improvements

### Recap

![](/files/-MT4cZeoePMCw5T0unpc)

## Now What?

![](/files/-MT4dSLA9YNz8Hrcy0tl)