Comm-to-comp Ratio

• Idea is to measure the volume of inherent communication per computation
  • In most cases it is beneficial to pick the decomposition with the lowest comm -to-comp ratio
  • But depends on the application structure i.e. picking the lowest comm -to-comp may have other problems
  • Normally this ratio gives you a rough estimate about average communication bandwidth requirement of the application i.e. how frequent is communication
• But it does not tell you the nature of communication i.e. bursty or uniform
• For grid solver comm. happens only at the start of each iteration; it is not uniformly distributed over computation
• Thus the worst case BW requirement may exceed the average comm -to-comp ratio

Extra Work

• Extra work in a parallel version of a sequential program may result from
  • Decomposition
  • Assignment techniques
  • Management of the task pool etc.
• Speedup is bounded above by Sequential work / Max (Useful work + Synchronization + Comm. cost + Extra work) where the Max is taken over all processors
• But this is still incomplete
  • We have only considered communication cost from the viewpoint of the algorithm and ignored the architecture completely