Overview of cache memories: 3 types of cache memories

There are 3 types of cache memories:

  1. Associative cache:

      • very flexible but very expensive to make.

  2. Direct-Mapped cache:

      • very inexpensive to make but not flexible at all.

  3. Set-Associative cache:

      • is built using the combination of the technique used to build associative cache and direct-mapped cache

      • it pretty flexible and yet economical

In this slide set, we study the associative cache

Review: computer memory

The program's view of the computer memory is:   each byte is identified by an address

(I will show you a different view - "data transfer view" - of the memory later...)

Review: data transfer between CPU and memory

  • The data bus of computers nowadays has at least 32 wires (64 wires are now common)

    Assumption:

    • In the discussion on cache memory, I will assume that the databus has 32 bits (= wires)

  • Remember that:

    • Each memory read operation will use all of the data bus wires

  • Therefore:

    • Each memory read operation will fetch 4 bytes from memory

      (Because I have assumed that the databus has 32 bits !)

    • The unit of data cached = 4 bytes (caches all bytes fetched !!)

Review: computer memory

The data transfer's view of the computer memory is as follows:   a unit of tranfer is 4 bytes

Each transfer (read) operation will transfer 4 bytes from memory to the CPU

Block number of an Associative Cache

The block number of an associative cache memory is a number that identifies a unit of data transfer:

How to find a block number for a given memory address:

  • For a given address X, the block number B that contains address X is:   B = X / 4

The structure of the associative cache

The associative cache memory has the following structure:

The Valid Flag is 1 bit

For a 32 bits address bus, the block number will be a 30 bit binary number (explained later)
(But I will use decimal numbers to represent its value)

For a 32 bits data bus, the value stored is a 32 bit binary number
(But I will use decimal numbers to represent it)

The structure of the associative cache
 

Definition:   slot of a cache memory = one row in a cache memory

Today, cache size ranges from 1 GBytes - 8 GBytes   or   0.25 G - 2 G slots

I will now explain how each field in the associative cache is used.

The structure of the associative cache
 

The valid flag indicates whether the value stored in cache entry is valid or invalid:

Initially:

  • The cache is empty and all entries are invalid !

When a slot is filled with data from memory, that slot will become valid

The structure of the associative cache
 

The block number identifies the memory locations that are cached in a cache entry:

 

For an address bus of 32 bits, the block number is a 30 bits binary number
I will denote block numbers as a (unsigned) decimal integer

(Why 30 bits will be explained later)

The structure of the associative cache

Recall how the block numbers are computed from memory addresses:   Blk Num = Addr / 4

 
 

The block numbers are stored in the cache to identify the memory locations being cached

The structure of the associative cache
 

Quiz: Can you tell which memory addresses are being cached in this cache entry:

 
 

Hint: block number 0 spans address 0,1,2,3, block number 1 spans address 4,5,6,7, ...

The structure of the associative cache
 

Answer:   data in memory addresses 16, 17, 18, 19   (= 4x4, 4x4+1, 4x4+2, 4x4+3)

 
 

Hint: block number 2 spans address 8,9,10,11, block number 3 spans address 12,13,14,15, ...

The structure of the associative cache
 

The value field contains the data from the 4 memory cells identified by the block number:

 
 

For a 32 bits data bus, the value is a 32 bits binary number
But I will denote it as a (unsigned) decimal integer

Example of the associative caching technique
 

The following diagram shows an example content of the memory and an associative cache:

 
 

Note: the memory content is shown in 4 bytes units and identified using block numbers !!!

Example of the associative caching technique
 

Example:   the slot with block # 78 is valid and contains the correct value stored in memory:

 
 

From the (1) valid field and (2) the block number field, the cache memory can tell that the value is from memory locations 78x4, 78x4+1, 78x4+2, 78x4+3

Cache operation (cache hit)

Suppose the CPU want to read the 4 bytes at address 16:

 
 

The cache memory will check whether the value is in the cache first before reading the RAM !!

Cache operation (cache hit)

The associative cache memory detects that it contains the correct value in one of its entry:

 
 

The cache memory will return the data to the CPU and no memory operation is performed !!

Cache operation (cache miss)

Suppose the CPU wnat to read the 4 bytes at address 408:

 
 

The cache memory will check whether the value is in the cache first before reading the RAM !!

Cache operation (cache miss)

The block # 102 is not found in any cache slot:

 
 

The cache memory will now perform a memory read operation and fetch the missing data from RAM

Cache operation (cache miss)

The data from memory will be inserted into a (unused) slot of the cache and then sent to the CPU:

 
 

Note:   if there is no unused slot, the slot that contains the "oldest" unused item is replaced