|
Multi-level caches
- Split-transaction bus makes the design of multi-level caches a little more difficult
- The usual design is to have queues between levels of caches in each direction
- How do you size the queues? Between processor and L1 one buffer is sufficient (assume one outstanding processor access), L1-to-L2 needs P+1 buffers (why?), L2-to-L1 needs P buffers (why?), L1 to processor needs one buffer
- With smaller buffers there is a possibility of deadlock: suppose the L1-to-L2 and L2-to-L1 have one queue entry each, there is a request in L1-to-L2 queue and there is also an intervention in L2-to-L1 queue; clearly L1 cannot pick up the intervention because it does not have space to put the reply in L1-to-L2 queue while L2 cannot pick up the request because it might need space in L2-to-L1 queue in case of an L2 hit
- Formalizing the deadlock with dependence graph
- There are four types of transactions in the cache hierarchy: 1. Processor requests (outbound requests), 2. Responses to processor requests (inbound responses), 3. Interventions (inbound requests), 4. Intervention responses (outbound responses)
- Processor requests need space in L1-to-L2 queue; responses to processors need space in L2-to-L1 queue; interventions need space in L2-to-L1 queue; intervention responses need space in L1-to-L2 queue
- Thus a message in L1-to-L2 queue may need space in L2-to-L1 queue (e.g. a processor request generating a response due to L2 hit); also a message in L2-to-L1 queue may need space in L1-to-L2 queue (e.g. an intervention response)
- This creates a cycle in queue space dependence graph
Dependence graph
- Represent a queue by a vertex in the graph
- Number of vertices = number of queues
- A directed edge from vertex u to vertex v is present if a message at the head of queue u may generate another message which requires space in queue v
- In our case we have two queues
- L2-L1 and L1-L2; the graph is not a DAG, hence deadlock
Multi-level caches
- In summary
- L2 cache controller refuses to drain L1-to-L2 queue if there is no space in L2-to-L1 queue; this is rather conservative because the message at the head of L1-to-L2 queue may not need space in L2-to-L1 queue e.g., in case of L2 miss or if it is an intervention reply; but after popping the head of L1-to-L2 queue it is impossible to backtrack if the message does need space in L2-to-L1 queue
- Similarly, L1 cache controller refuses to drain L2-to-L1 queue if there is no space in L1-to-L2 queue
- How do we break this cycle?
- Observe that responses for processor requests are guaranteed not to generate any more messages and intervention requests do not generate new requests, but can only generate replies
- Solving the queue deadlock
- Introduce one more queue in each direction i.e. have a pair of queues in each direction
- L1-to-L2 processor request queue and L1-to-L2 intervention response queue
- Similarly, L2-to-L1 intervention request queue and L2-to-L1 processor response queue
- Now L2 cache controller can serve L1-to-L2 processor request queue as long as there is space in L2-to-L1 processor response queue, but there is no constraint on L1 cache controller for draining L2-to-L1 processor response queue
- Similarly, L1 cache controller can serve L2-to-L1 intervention request queue as long as there is space in L1-to-L2 intervention response queue, but L1-to-L2 intervention response queue will drain as soon as bus is granted
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|