|
Caching shared data?
- All transactions are no longer visible to all
- Whether a page should be cached or not is normally part of the VA to PA translation
- For example, in some graphics co-processors all operations must be through uncached writes or storing command/data to memory-mapped control registers is also uncached
- Private memory lines can be cached without any problem and does not depend on if the line is local or remote
- Caching shared lines introduce coherence issues
COWs and NOWs
- Historically, used to build a multi-programmed multi-user system
- Connect a number of PCs or workstations over a cheap commodity network and schedule independent jobs on machines depending on the load
- Increasingly, these clusters are being used as parallel machines
- One major reason is the availability of message passing libraries to express parallelism over commodity LAN or WAN
- Also, technology breakthrough in terms of high-speed interconnects (ServerNet, Myrinet, Infiniband, PCI Express AS, etc.)
- Varying specialized support in CA
- Conventional TCP/IP stack imposes an enormous overhead to move even a small amount of data (often more than common Ethernet): network processor architecture has been a hot research topic
- Active messages allow user-level communication
- Reflective memory allows writes to special regions of memory to appear as writes into regions on remote processors
- Virtual interface architecture (VIA): each process has a communication end point consisting of a send queue, receive queue, and status; a process can deposit a message in its send queue with a dest. id so that it actually gets into the receive queue of target process
- The CA hardware normally plugs on to I/O bus as opposed memory bus (fast PCI bus supports coherence); could be on the graphics bus also
Scalable synchronization
- In message-passing a send/receive pair provides point-to-point synchronization
- Handle all-to-all synchronization via tree barrier
- Also, all-to-all communication must be properly staggered to avoid hot-spots in the system
- Classical example of matrix transpose
- Scalable locks such as ticket or array should be used
- Any problem with array locks?
- Array locations now may not be local: invalidation causes remote misses
Distributed queue locks
- Goodman, Vernon, Woest (1989)
- Logically arrange the array as a linked list
- Allocate a new node (must be atomic) when a processor enters acquire
- The node is allocated on the local memory of the contending processor
- A tail pointer is always maintained
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|