Module 7: "Parallel Programming"
  Lecture 12: "Steps in Writing a Parallel Program"
 

Mapping

  • At this point you have a parallel program
    • Just need to decide which and how many processes go to each processor of the parallel machine
  • Could be specified by the program
    • Pin particular processes to a particular processor for the whole life of the program; the processes cannot migrate to other processors
  • Could be controlled entirely by the OS
    • Schedule processes on idle processors
    • Various scheduling algorithms are possible e.g., round robin: process#k goes to processor#k
    • NUMA-aware OS normally takes into account multiprocessor-specific metrics in scheduling
  • How many processes per processor? Most common is one-to-one

An example

  • Iterative equation solver
    • Main kernel in Ocean simulation
    • Update each 2-D grid point via Gauss-Seidel iterations
    • A[i,j] = 0.2(A[i,j]+A[i,j+1]+A[i,j-1]+A[i+1,j]+A[i-1,j])
    • Pad the n by n grid to (n+2) by (n+2) to avoid corner problems
    • Update only interior n by n grid
    • One iteration consists of updating all n2 points in-place and accumulating the difference from the previous value at each point
    • If the difference is less than a threshold, the solver is said to have converged to a stable grid equilibrium

Sequential program

int n;
float **A, diff;

begin main()
   read (n);   /* size of grid */
   Allocate (A);
   Initialize (A);
   Solve (A);
end main

  begin Solve (A)
     int i, j, done = 0;
     float temp;
     while (!done)
        diff = 0.0;
      for i = 0 to n-1
            for j = 0 to n-1
               temp = A[i,j];
               A[i,j] = 0.2(A[i,j]+A[i,j+1]+A[i,j-1]+A[i-1,j]+A[i+1,j]);
               diff += fabs (A[i,j] - temp);
            endfor
        endfor
        if (diff/(n*n) < TOL) then done = 1;
   endwhile
 end Solve