Module 7: "Parallel Programming"
  Lecture 13: "Parallelizing a Sequential Program"
 

Message passing

  • What is different from shared memory?
    • No shared variable: expose communication through send/receive
    • No lock or barrier primitive
    • Must implement synchronization through send/receive
  • Grid solver example
    • P0 allocates and initializes matrix A in its local memory
    • Then it sends the block rows, n, P to each processor i.e. P1 waits to receive rows n/P to 2n/P-1 etc. (this is one-time)
    • Within the while loop the first thing that every processor does is to send its first and last rows to the upper and the lower processors (corner cases need to be handled)
    • Then each processor waits to receive the neighboring two rows from the upper and the lower processors
  • At the end of the loop each processor sends its local_diff  to P0 and P0 sends back the done  flag

Major changes