Virtual networks

• Consider a two-node system with one incoming and one outgoing queue on each node
• Single queue is not enough to avoid deadlock
  • Single queue forms a single virtual network
• Similar deadlock issues as multi-level caches
  • An incoming message may generate another message e.g., request generates reply, ReadX generates reply and invalidation requests, request may generate intervention request
  • Memory controller refuses to schedule a message if the outgoing queue is full
  • Same situation may happen on all nodes: deadlock
  • One incoming and one outgoing queue is not enough
  • What if we have two in each direction?: one for request and one for reply
  • Replies can usually sink
  • Requests generating requests?
• What is the length of the longest transaction in terms of number of messages?
  • This decides the number of queues needed in each direction (Origin 2000 uses a different scheme)
  • One type of message is usually assigned to a queue
  • One queue type connected across the system forms a virtual network of that type e.g. request network, reply network, third party request (invalidations and interventions) network
  • Virtual networks are multiplexed over a physical network
• Sink message type must get scheduled eventually
  • Resources should be sized properly so that scheduling of these messages does not depend on anything
  • Avoid buffer shortage (and deadlock) by keeping reserved buffer for the sink queue

Three-lane protocols

• Quite popular due to its simplicity
  • Let the request network be R, reply network Y, intervention/invalidation network be RR
  • Network dependence (aka lane dependence) graph looks something like this