How caches work
Memory hierarchy: caches

The use of such a memory hierarchy is based on the non-uniform nature of instruction and data accesses in most computer programs, sometimes summarized as the 10%-90% rule: a minority of instructions execute for the majority of the time, and the majority of instructions account for only a small proportion of execution time. This non-uniformity of memory accesses is also known as the principle of locality, which manifests itself in two ways.

– The first aspect is spatial: when a memory reference intervenes, there's a good chance that the next memory reference will be for a neighboring address. Instructions are sequenced between branches. There are program-specific zones in which to place data. In addition, elements of one-dimensional...