Hard Drive Penalties
In an introductory programming class, I had a professor who explained CPU cache as sitting in a library and having the book in front of you. RAM was like going to a bookshelf. And hitting the hard drive was like ordering it from a store and waiting for it to come.
While those numbers aren't exactly accurate, it does communicate the importance of keeping your data as close to the processor as possible. I recently came across an interesting study by Diomidis Spinellis that shows the relative speeds of different types of memory.
| Nominal | Worst case | Sustained | Productivity | |||
| Component | size | latency | throughput | $1 buys | (Bytes read / s / $) | |
| (MB/s) | Worst case | Best case | ||||
| L1 D cache | 64 KB | 1.4ns | 19022 | 10.7 KB | 7.91·1012 | 2.19·1014 |
| L2 cache | 512 KB | 9.7ns | 5519 | 12.8 KB | 1.35·1012 | 7.61·1013 |
| DDR RAM | 256 MB | 28.5ns | 2541 | 9.48 MB | 3.48·1014 | 2.65·1016 |
| Hard drive | 250 GB | 25.6ms | 67 | 2.91 GB | 1.22·1011 | 2.17·1017 |
If you have small chunks of data, the latency difference is huge. L1 cache is roughly 55,000 times faster than the hard drive. If you're dealing with sustained throughput, hitting L1 cache is 283.9 times faster than going to the hard drive.
Draw your own conclusions here, but keep this in mind when you're writing apps. (Note that the cost of memory is a bit outdated in this chart.)