Message Passing

• Actual implementation of message layer
  • Initially it was very topology dependent
  • A node could talk only to its neighbors through FIFO buffers
  • These buffers were small in size and therefore while sending a message send would occasionally block waiting for the receive to start reading the buffer (synchronous message passing )
  • Soon the FIFO buffers got replaced by DMA (direct memory access) transfers so that a send can initiate a transfer from memory to I/O buffers and finish immediately (DMA happens in background); same applies to the receiving end also
  • The parallel algorithms were designed specifically for certain topologies: a big problem

• To improve usability of machines, the message layer started providing support for arbitrary source and destination (not just nearest neighbors)
  • Essentially involved storing a message in intermediate “hops” and forwarding it to the next node on the route
  • Later this store-and-forward routing got moved to hardware where a switch could handle all the routing activities
  • Further improved to do pipelined wormhole routing so that the time taken to traverse the intermediate hops became small compared to the time it takes to push the message from processor to network (limited by node-to-network bandwidth )
  • Examples include IBM SP2, Intel Paragon
  • Each node of Paragon had two i860 processors, one of which was dedicated to servicing the network (send/ recv . etc.)