|
Page migration
- Page migration changes the existing VA to PA mapping of the migrated page
- Requires notifying all TLBs caching the old mapping
- Introduces a TLB coherence problem
- Origin 2000 uses a smart page migration algorithm: allows the page copy and TLB shootdown to proceed in parallel
- Array of 64 page reference counters per directory entry to decide whether to migrate a page or not: compare requester’s counter against home’s and send an interrupt to home if migration is required
- What does the interrupt handler do?
- Access all directory entries of the lines belonging to the to-be migrated page
- Send invalidations to sharers or interventions to owners; at the end all cache lines of that page must be in memory
- Set the poison bits in the directory entries of all the cache lines of the page
- Start a block transfer of the page from home to requester at this point (30 μs to copy 16 KB)
- An access to a poisoned cache line from a node results in a bus error which invalidates the TLB entry for that page in the requesting node (avoids broadcast shootdown)
- Until the page is completely migrated and is assigned a physical page frame on target node, all nodes accessing a poisoned line wait in a pending queue
- After the page copy is completed the waiting nodes are served one by one; however, the directory entries and the page itself are moved to a “poisoned list” and are not yet freed at the home (i.e. you still cannot use that physical page frame)
- On every scheduler tick the kernel invalidates one TLB entry per processor
- After a time equal to TLB entries per processor multiplied by scheduling quantum the page frame is marked free and is removed from the poisoned list
- Major advantage: requesting nodes only see the page copy latency including invalidation and interventions in critical path, but not the TLB shootdown latency
Queue lock in hardware
- Stanford DASH
- Memory controller recognizes lock accesses
- Requires changes in compiler and instruction set
- Marks the directory entry with contenders
- On unlock a contender is chosen and lock is granted to that node
- Unlock is forced to generate a notification message to home
- Possibly requires special cache state for lock variables or special uncached instructions for unlock if lock variables are not allowed to be cached
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|