Rui's Blog

Lecture 7: Parallel Computing. Flynn's Taxonomy. Amdahl's Law.

Lecture Summary

  • Wrap up Virtual Memory
  • Intuitions for Parallel Computing
  • Flynn's Taxonomy
  • Amdahl's Law

Why Parallel Computing?

Sequential computing is facing these steep hills to climb:
  • Memory Wall: Speed difference between CPU & memory outside the chip
  • ILP Wall
  • Power Wall: Latency & limited communication bandwidth beyond chip boundaries

Memory Wall

Take-home message: Try to stay away from long and winding conversations with the main memory

ILP Wall

ILP elicits very complex microarchitecture
Instruction pipelining; Superscalar execution; Out-of-order execution; Register renaming; Speculative execution; Branch prediction
Predicting the future comes at the cost of microarchitecture complexity and power cost

Power Wall

Power, and not manufacturing, limits traditional general-purpose microarchitecture improvements


Now What?