Back to shared bus

• Centralized shared bus makes it easy to support SC
  • Writes and reads are all serialized in a total order through the bus transaction ordering
  • If a read gets a value of a previous write, that write is guaranteed to be complete because that bus transaction is complete
  • The write order seen by all processors is the same in a write through system because every write causes a transaction and hence is visible to all in the same order
  • In a nutshell, every processor sees the same total bus order for all memory operations and therefore any bus-based SMP with write through caches is SC
• What about a multiprocessor with writeback cache?
  • No SMP uses write through protocol due to high BW

Snoopy protocols

• No change to processor or cache
  • Just extend the cache controller with snoop logic and exploit the bus
• We will focus on writeback caches only
  • Possible states of a cache line: Invalid (I), Shared (S), Modified or dirty (M), Clean exclusive (E), Owned (O); every processor does not support all five states
  • E state is equivalent to M in the sense that the line has permission to write, but in E state the line is not yet modified and the copy in memory is the same as in cache; if someone else requests the line the memory will provide the line
  • O state is exactly same as E state but in this case memory is not responsible for servicing requests to the line; the owner must supply the line (just as in M state)
  • Stores really read the memory (as opposed to write)

Stores

• Look at stores a little more closely
  • There are three situations at the time a store issues: the line is not in the cache, the line is in the cache in S state, the line is in the cache in one of M, E and O states
  • If the line is in I state, the store generates a read-exclusive request on the bus and gets the line in M state
  • If the line is in S or O state, that means the processor only has read permission for that line; the store generates an upgrade request on the bus and the upgrade acknowledgment gives it the write permission (this is a data-less transaction)
  • If the line is in M or E state, no bus transaction is generated; the cache already has write permission for the line (this is the case of a write hit; previous two are write misses)

Invalidation vs. update

• Two main classes of protocols:
  • Invalidation-based and update-based
  • Dictates what action should be taken on a write
  • Invalidation-based protocols invalidate sharers when a write miss (upgrade or readX) appears on the bus
  • Update-based protocols update the sharer caches with new value on a write: requires write transactions (carrying just the modified bytes) on the bus even on write hits (not very attractive with writeback caches)
  • Advantage of update-based protocols: sharers continue to hit in the cache while in invalidation-based protocols sharers will miss next time they try to access the line
  • Advantage of invalidation-based protocols: only write misses go on bus (suited for writeback caches) and subsequent stores to the same line are cache hits

Which one is better?

• Difficult to answer
  • Depends on program behavior and hardware cost
• When is update-based protocol good?
  • What sharing pattern? (large-scale producer/consumer)
  • Otherwise it would just waste bus bandwidth doing useless updates
• When is invalidation-protocol good?
  • Sequence of multiple writes to a cache line
  • Saves intermediate write transactions
• Also think about the overhead of initiating small updates for every write in update protocols
  • Invalidation-based protocols are much more popular
  • Some systems support both or maybe some hybrid based on dynamic sharing pattern of a cache line