More Synchronization

• Global synchronization
  • Through barriers
  • Often used to separate computation phases
• Point-to-point synchronization
  • A process directly notifies another about a certain event on which the latter was waiting
  • Producer-consumer communication pattern
  • Semaphores are used for concurrent programming on uniprocessor through P and V functions
  • Normally implemented through flags on shared memory multiprocessors (busy wait or spin)

P 0 : A = 1; flag = 1;
P 1 : while (!flag); use (A);

Message Passing

• What is different from shared memory?
  • No shared variable: expose communication through send/receive
  • No lock or barrier primitive
  • Must implement synchronization through send/receive
• Grid solver example
  • P 0 allocates and initializes matrix A in its local memory
  • Then it sends the block rows, n, P to each processor i.e. P 1 waits to receive rows n/P to 2n/P-1 etc. (this is one-time)
  • Within the while loop the first thing that every processor does is to send its first and last rows to the upper and the lower processors (corner cases need to be handled)
  • Then each processor waits to receive the neighboring two rows from the upper and the lower processors
• At the end of the loop each processor sends its local_diff to P 0 and P 0 sends back the accumulated diff so that each processor can locally compute the done flag