# Lecture 7: Parallel Computing. Flynn's Taxonomy. Amdahl's Law.

## Lecture Summary

* Wrap up Virtual Memory
* Intuitions for Parallel Computing
* Flynn's Taxonomy
* Amdahl's Law

## Why Parallel Computing?

Sequential computing is facing these steep hills to climb:

* Memory Wall: Speed difference between CPU & memory outside the chip
* ILP Wall
* Power Wall: Latency & limited communication bandwidth beyond chip boundaries

### Memory Wall

![](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MT4UUBWaEzRcX3Qdtni%2F-MT4a2iWQ7l99EZ7X4TY%2FScreen%20Shot%202021-02-09%20at%201.01.20%20AM.png?alt=media\&token=a43c838c-c722-4146-8394-96fd7f437bcb)

Take-home message: Try to stay away from long and winding conversations with the main memory

### ILP Wall

![ILP elicits very complex microarchitecture](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MT4UUBWaEzRcX3Qdtni%2F-MT4axpClWcbfFSwnde4%2FScreen%20Shot%202021-02-09%20at%201.05.15%20AM.png?alt=media\&token=396b1af7-e2d9-4273-89e0-651be65684f4)

Instruction pipelining; Superscalar execution; Out-of-order execution; Register renaming; Speculative execution; Branch prediction

Predicting the future comes at the cost of microarchitecture complexity and power cost

### Power Wall

Power, and not manufacturing, limits traditional general-purpose microarchitecture improvements

### Recap

![](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MT4UUBWaEzRcX3Qdtni%2F-MT4cZeoePMCw5T0unpc%2FScreen%20Shot%202021-02-09%20at%201.12.19%20AM.png?alt=media\&token=1c2b2250-579a-4c55-8b9f-9e37f61f5fda)

## Now What?

![](https://1313833672-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MMTslgmrrtRXvxD2lk9%2F-MT4UUBWaEzRcX3Qdtni%2F-MT4dSLA9YNz8Hrcy0tl%2FScreen%20Shot%202021-02-09%20at%201.16.11%20AM.png?alt=media\&token=ec1f5e57-f984-42c9-8e0c-e1ab9db651ff)