Module 10: Open Multi-Processing
  Lecture 19: What is Parallelization?
 


What is Data Race ?

  • When two or more different threads in a multithreaded shared memory model access the same memory location the program
    may produce unexpected results
  • Data race occurs under following conditions:
    • There are two or more different threads accessing the same memory location concurrently
    • They don’t host any locks
    • At least one access is write

A “for” Loop

“for” Loop

for(int i = 0 ; i < 8 ; i++)
a[i] = a[i] + b[i];

Execution in Parallel With 2 Threads

Thread 1
a[0] = a[0] + b[0]
a[1] = a[1] + b[1]
a[2] = a[2] + b[2]
a[3] = a[3] + b[3]
Thread 2
a[4] = a[4] + b[4]
a[5] = a[5] + b[5]
a[6] = a[6] + b[6]
a[7] = a[7] + b[7]

Overview to OpenMP

What is OpenMP ?

An API that may be used to explicitly direct multi-threaded, shared memory parallelism.

When to Use OpenMP For Parallelism ?

  • A loop is not parallelized
    The data dependence analysis is not able to determine whether it is safe to parallelize or not
  • The Granularity is not enough
    The compiler lacks information to parallelize at highest possible level