Lecture 7: Parallel Computing. Flynn's Taxonomy. Amdahl's Law.
PreviousLecture 6: The Walls to Sequential Computing. Moore’s Law.NextLecture 8: GPU Computing Intro. The CUDA Programming Model. CUDA Execution Configuration.
Last updated
Last updated
Wrap up Virtual Memory
Intuitions for Parallel Computing
Flynn's Taxonomy
Amdahl's Law
Sequential computing is facing these steep hills to climb:
Memory Wall: Speed difference between CPU & memory outside the chip
ILP Wall
Power Wall: Latency & limited communication bandwidth beyond chip boundaries
Take-home message: Try to stay away from long and winding conversations with the main memory
Instruction pipelining; Superscalar execution; Out-of-order execution; Register renaming; Speculative execution; Branch prediction
Predicting the future comes at the cost of microarchitecture complexity and power cost
Power, and not manufacturing, limits traditional general-purpose microarchitecture improvements
