CS355 Sylabus

Introduction to the Virtual Memory technique

So far, your concept of how a program is executed is (probably) as follows:

The entire program is placed in memory before the CPU even fetches the first instruction and execute it....
In other words:
- The address sepcified by the program is also the address in memory.
Well, your view is about to be shattered into oblivion... :-)

Myth #1: the entire program needs to be in memory

Fact:

To execute the current program (assembler) instruction , you need the data present (= stored) in memory:

The instruction pointed to by the program counter (so the CPU can fetch the instruction from memory)
All the memory variables used by the instruction (so the CPU can fetch/store value to these variables

All other memory locations are unused when the CPU executes the current instruction !!!

Myth #2: the program address is the same as a memory address
- Program address:
- Memory address:
- Fact:
  so you don't have to worry about how the program was stored in memory
Postscript
- Fact:
  So in software courses, you do not have the need to know....
What's wrong with the old (and erroneous) view:
- Taking the view that an address sepcified by the program is the same value as an address in memory will limit the program size to the physical size of the memory.
- Believe it or not, but the ability of a computer program to specify how large the program memory is, has absolutely nothing to do with how big the computer memory is in reality....
The above statement needs some clarification...
Suppose I give you an assembler instruction of a hypothetical machine than has the following format:

The opcode encodes the instruction code, and the 3 operand fields encode the address of the operand in memory.
Now a computer program contains instructions - such as the one in the figure above.
The instructions use variables in memory whose address is given by the values in the different operand fields.
The memory locations referenced by the program instructions are called program memory.
The ability of a computer program to specify how large the program memory is depends on how many bits the instruction uses to specify the address.
So the ability of a computer program to specify how large the program memory is, has absolutely nothing to do with how big the physical computer memory is in reality....
On the other hand, how large the physical memory depends on how many DIMMs you has purchase at your nearest computer outlet store...
But you may ask:
- What good is it to have the program use addresses that is larger than the physical memory ?
- If you have 64 Mbytes of physical memory, can your program use more than 64 Mbytes ?
Before the year 1961 (before most of you are even born), the above statement would have been true...
Before 1961, the address encoded in a program instruction is also the address in the physical memory, in every single computer.
But in 1961, researchers in Manchester, England, proposed a novel idea to separate the concepts of program address and (physical) memory address.
They made an astute obversvation that an address used in a program identifies a location within the program, while an address that identifies a memory location can be separated
Sure, the data referenced by a program address must be stored somewhere in the computer memory (afterall, you have to return the data), but it is not necessary to store the data in the memory location that is given by the program address !!!
They proposed the following:
- They define a dynamic memory mapping function that maps (translates) a program address to a memory address.
The device that provides the dynamic memory mapping function nowadays is called the Memory Management Unit (MMU) (it used to be software in memory back in 1961 but it is a chip nowadays):
- As you may know from Mathematics, a mapping relates an (input) value from the domain to an (output) value from a co-domain.
- Note that this mapping is dynamic, i.e., the mapping function changes over time.
  The fact that the map is dynamic is crucial in understanding how this technqiue works.
Before going into the details of this technique, let's clear up some jargons:
- The (program) address space is the set of all possible addresses that can be used by a program. (When you hear a computer scientist talk about "address space", he means the program address space).
  The program address space is also called the virtual address space.
  The address space depends on the instruction format - specifically, the number of bits is used in the instruction to encode a memory operand.
- The physical address space is the set of addresses that is available in a computer system.
  The set of address in this set depends on how many memory chips you have installed.
- The technique using a dynamic memory mapping to translate program address to physical addresses is called Virtual Memory.
- For example, M68000 instructions use 32 bits to encode a memory that is located in memory. So the program address space consists of the addresses [0..2³²-1].
- When the M68000 CPU was sold (back in around 1981), then physical memory used in PCs were around 128 Kbytes...
  And yet, a program written in M68000 assembler code can use all 2³² addresses !
How is it possible that a program can access much more memory locations than there are physically available ?
The answer to that is that when a program is run, you don't need all parts of the program to be present inside the physical memory, but only the parts that are being accessed (and that portion is a very small part of the whole program).
In order to execute a program instruction, you need to obtain everything that is needed to complete the execution.
These things are:
- The instruction itself (ofcourse)
- The source operand(s) used by the instruction. These operands can come from the registers or the memory (variables are stored in memory)
- The destination operand used by the instruction. The destination operand can be one of the registers or a memory location (variables).
In order to allow a program to execute, the virtual memory technique only needs to map:
- the portion of the program addresses that contains the next instruction (at program address given by the value in the program counter).
- the portion of the program addresses that contains the memory operands (variables) used by the next instruction.
Example:
The following figure shows the mamory mapping function when a program first starts execution:
- The portion of the program that contains the next instruction resides in physical memory.
- The variables used by the current instructions are also in physical memory.
- The memory mapping is setup in such way that it maps the program portion to the first part of the physical memory and the variable portion to the bottom part of the physical memory.
When the program runs for a while and enters a different part of the program, instruction in the first part of the program is usually not used by the second part of the program (at least not for a long time) !
You can recycle the physical memory that was used to store instructions in the first part of the program and use the same portion of the physical memory to store instruction in the second portion of the program:
- Again, the portion of the program that contains the next instruction resides in physical memory.
- The variables used by the current instructions are also in physical memory.
- The memory mapping must by re-setup in such way that it maps the second portion of the program to the first part of the physical memory !!!
You still need to have a copy of the whole program somewhere so that you can get the different parts of the program to place in memory.
The place that stores the entire program is the hard disk.
Operational summary of the virtual memory technique:
- The entire program is stored on disk.
- Portion(s) (called "pages") of the program that are needed immediately to execute the next instruction is dynamically fetched from disk and placed in the memory.
- When a pages are brought in, the memory mapping function is updated to reflect the new mapping of program addresses to memory addresses.
- When a new page brought in from disk replaces some existing page in memory, the page in memory must first be written back to disk ! (Because the page may contains variables that have been updated by the program !)
In order for the virtual memory technique to work well, the memory mapping function must be able to be performed efficiently...
We will next see how this mapping is realised...