Lecture Summary
Intuitions for Parallel Computing
Why Parallel Computing?
Sequential computing is facing these steep hills to climb:
Memory Wall: Speed difference between CPU & memory outside the chip
Power Wall: Latency & limited communication bandwidth beyond chip boundaries
Memory Wall
Take-home message: Try to stay away from long and winding conversations with the main memory
ILP Wall
Instruction pipelining; Superscalar execution; Out-of-order execution; Register renaming; Speculative execution; Branch prediction
Predicting the future comes at the cost of microarchitecture complexity and power cost
Power Wall
Power, and not manufacturing, limits traditional general-purpose microarchitecture improvements
Recap
Now What?