Module 3: Fundamentals of Parallel Computers: ILP vs TLP
  Lecture 5: Communication Architectures and Communication Costs
 


Design Issues

  • Need to understand architectural components that affect software
    • Compiler , library, program
    • User/system interface and hw/ sw interface
    • How programming models efficiently talk to the communication architecture?
    • How to implement efficient primitives in the communication layer?
    • In a nutshell, what issues of a parallel machine will affect the performance of the parallel applications?
  • Naming, Operations, Ordering, Replication, Communication cost

Naming

  • How are the data in a program referenced?
    • In sequential programs a thread can access any variable in its virtual address space
    • In shared memory programs a thread can access any private or shared variable (same load/store model of sequential programs)
    • In message passing programs a thread can access local data directly
  • Clearly, naming requires some support from hw and OS
    • Need to make sure that the accessed virtual address gets translated to the correct physical address

Operations

  • What operations are supported to access data?
    • For sequential and shared memory models load/store are sufficient
    • For message passing models send/receive are needed to access remote data
    • For shared memory, hw (essentially the CA) needs to make sure that a load/store operation gets correctly translated to a message if the address is remote
    • For message passing, CA or the message layer needs to copy data from local memory and initiate send, or copy data from receive buffer to local memory