Design issues

• Need to understand architectural components that affect software
  • Compiler, library, program
  • User/system interface and hw/sw interface
  • How programming models efficiently talk to the communication architecture?
  • How to implement efficient primitives in the communication layer?
  • In a nutshell, what issues of a parallel machine will affect the performance of the parallel applications?
• Naming, Operations, Ordering, Replication, Communication cost

Naming

• How are the data in a program referenced?
  • In sequential programs a thread can access any variable in its virtual address space
  • In shared memory programs a thread can access any private or shared variable (same load/store model of sequential programs)
  • In message passing programs a thread can access local data directly
• Clearly, naming requires some support from hw and OS
  • Need to make sure that the accessed virtual address gets translated to the correct physical address

Operations

• What operations are supported to access data?
  • For sequential and shared memory models load/store are sufficient
  • For message passing models send/receive are needed to access remote data
  • For shared memory, hw (essentially the CA) needs to make sure that a load/store operation gets correctly translated to a message if the address is remote
  • For message passing, CA or the message layer needs to copy data from local memory and initiate send, or copy data from receive buffer to user area in local memory

Ordering

• How are the accesses to the same data ordered?
  • For sequential model, it is the program order: true dependence order
  • For shared memory, within a thread it is the program order, across threads some “valid interleaving” of accesses as expected by the programmer and enforced by synchronization operations (locks, point-to-point synchronization through flags, global synchronization through barriers)
  • Ordering issues are very subtle and important in shared memory model (some microprocessor re-ordering tricks may easily violate correctness when used in shared memory context)
  • For message passing, ordering across threads is implied through point-to-point send/receive pairs (producer-consumer relationship) and mutual exclusion is inherent (no shared variable)