CPU performance: cache size

What the CPU does every time it needs data

Follow one memory request through the hierarchy — notice where cache size makes the difference.

1. 1CPU requests dataEvery instruction the CPU executes may need data from memory. The CPU generates a memory address and begins checking the fastest store it has access to.+
2. 2Check cacheCache is checked first because it is fastest. If the data is there, the CPU retrieves it quickly and carries on.+
3. 3Check the further cache levelsIf the data is not found in the nearest cache, the CPU checks further cache levels. Each is slower to reach but larger — and any cache hit still saves a slow trip to main memory.+
4. 4Cache miss — fetch from RAMIf the data is in no cache, the CPU must fetch it from main memory, which is much slower. The CPU waits and cannot do useful work. This is the delay a larger cache helps avoid.+
5. 5Data loaded into cacheOnce fetched from RAM, the data is loaded into cache so that the next access to the same address is a cache hit. Larger cache means more of this recently used data can be retained.+

Predict, then check

Think about what 'more cache' actually changes before you reveal the answer.

A CPU is running a large video-editing program and spends a lot of time waiting for data. Explain how increasing the cache size could reduce that waiting.