Communication Cost

• Better model:
  • Communication time for n bytes = Overhead + CA occupancy + Network latency + Size/BW + Contention
  • T(n) = Ov + Oc + L + n/B + Tc
  • Overhead and occupancy may be functions of n
  • Contention depends on the queuing delay at various components along the communication path e.g. waiting time at the communication assist or controller, waiting time at the router etc.
  • Overall communication cost = frequency of communication x (communication time – overlap with useful computation)
  • Frequency of communication depends on various factors such as how the program is written or the granularity of communication supported by the underlying hardware

ILP vs. TLP

• Microprocessors enhance performance of a sequential program by extracting parallelism from an instruction stream (called instruction-level parallelism)
• Multiprocessors enhance performance of an explicitly parallel program by running multiple threads in parallel (called thread-level parallelism)
• TLP provides parallelism at a much larger granularity compared to ILP
• In multiprocessors ILP and TLP work together
  • Within a thread ILP provides performance boost
  • Across threads TLP provides speedup over a sequential version of the parallel program