---

Motivation of using a cache

---

• How a computer uses a cache memory
  • Using a cache:

    The cache is initially empty When the CPU fetches: Instructions         or           Operands (variables) they from memory: The instruction and operations are stored in the cache for future use When the CPU fetches from the same memory location again: The value will now be found inside the cache !!! In this case: The cache will return the value to the CPU directly (Without having to fetch the data from memory)

  ---

  ---

  ---

  ---

• Demonstrating the usefulness of a cache
  • Consider a program loop that consist of 100 instructions:

    mov r0, #0 Loop: cmp r0, #1000 <----+ beq Done | instr 1 | 100 instructions instr 2 | ... | Each instruction is store add r0, r0, #1 | in 1 memory location (for simplicity) b Loop <----+ Done: ...

    The loop is executed for 1000 times

    ---

    ---

  • Suppose:

    Memory speed = 50 nsec to fetch one instruction Cache speed = 5 nsec to fetch one instruction

    ---

    ---

  • Time needed to fetch instructions to execute the loop without using cache memory:

    # instructions fetched = 1000 × 100 Time needed to fetch 1 instr = 50 nsec Total time needed to fetch insr = 50 × 1000 × 100 = 5,000,000 nsec Avg time = 5,000,000 / 100000 = 50 nsec per instruction

    ---

    ---

  • Time needed to fetch instructions to execute the loop using a cache memory:

    First time throught the loop: # instructions fetched = 1 × 100 Time needed to fetch 1 instr = 50 nsec (from memory) Total time needed = 50 × 1 × 100 = 5000 nsec Second, 3rd, ... 1000 time throught the loop: # instructions fetched = 999 × 100 Time needed to fetch 1 instr = 5 nsec (from cache !!!) Total time needed to fetch insr = 5 × 999 × 100 = 499,500 nsec Grand total = 5000 + 499,500 = 504500 nsec Avg time = 504500 /100000 = 5.045 nsec per instruction !!!

  ---

  ---

  ---

  ---

• Cache hit and cache hit ration
  • Cache hit:

    Cache hit = a memory request that can be satisfied by the cache (I.e., the cache contains the requested value)

  • Cache hit ratio:

    Cache hit ratio = the fraction of memory requests that were satisfied by the cache

    ---

    ---

  • Example:

    The execution of the above program using a cache: # instruction fetched from memory = 1 × 100 = 100 # instruction fetched from cache = 999 × 100 = 99900 Total # instructions fetched = 100 + 99900 = 100,000 99900 Cache hit ratio = ---------- = 0.999 = 99.9 % 100000

    ---

    ---

  • Comment:

    Experimental data reports that a typical computer program execution has Cache hit ratio of 90% - 95% (Depending on how often the program loops)