CS355 Sylabus

Solving ALU instruction Data Hazard in the Basic Pipelining

Data Forwarding

The ALU instruction data hazard is caused by:

Possible solutions:

Update the register faster
We keep the pipeline structure;
but we add circuits to the pipeline to make the new value of the register available to the following instruction without having updated the destination register

Note:

The solution that we will discussed next is similar to caching !!!!
(We will "cache" the value of an updated register so that we can retrieve its value !!!)

Data Forwarding Hardware for ALU instructions

The following figure shows the new/modified circuits used to implement the data forwarding technique (the new/modified circuits are colored):

We need circuits to perform these functions:

Store the new values of the updated registers (and identify which register(s) are being updated)
Check if a value in an updated register is used before the register has been updated

These circuitry are added to store the new values of the updated registers (and to identify the updated registers):

2 Forwarding Registers (made with D-flipflops) named FR1 and FR2.
2 Tag Registers (made with D-flipflops) named tag1 and tag2.

Using the tag + forwarding registers:

The values in the tag + forwarding registers are used to indicate which register will be updated later with what value
Example: suppose we see the following tag + forwarding values:
This means:

Connections:

The destination address field in the IR(EX) register is connected to the input of the tag1 register (this will identify which register will be updated)
The ALU output is connected to the input of the forwarding register FR1 (the new value will be stored in FR1)

The output of the tag1 register is connected to the input of the tag2 register

The output of the forwarding register FR1 is connected to the input of the forwarding register FR2

This connection will copy the tag1 + FR1 register into the tag2 + FR2 register
This is done to keep the 2 lastest updated values + register number in the data forwarding registers "cache" !!!

(Recall from this example execution -- click here -- that 2 instructions following add r1,r2,r3 cannot obtain the new values)

The multiplexors in the EX stage is modified to direct the new values in the forwarding registers (FR1 and FR2) to the inputs of the ALU (so that the ALU can use these new value when appropriate !!!)

The multiplexor that selects the first input of the ALU:

The first multiplexor (colored yellow) will select from among: PC1 (contains the PC for a branch instruction), the "A"-register, Forwarding Register FR1 and Forwarding Register FR2

The selection logic of the first multiplexor is as follows:

if ( Instruction == Branch ) select PC1 as output operand else if ( tag1 == Src1 ) select Forward Register FR1 as output operand else if ( tag2 == Src1 ) select Forward Register FR2 as output operand else select the "A"-register as output operand

The reason for this logic is because:

A branch instruction does not use any general purpose registers and will always use register PC1 (which contains the program counter) as first operand.
Therefore, if we detects a branch instruction, the first multiplexor will output PC1 for the ALU

If the instruction is an ALU, Load or Store isntruction (= a non-branch instruction), then: (the instruction can use values from a general purpose register !!!):

If the source register 1 operand is the register updated by the last instruction (detected by tag1 == Src1 ), then we must use the value stored in the Forwarding Register FR1 as input operand for the ALU !!!
Otherwise, if the source register 1 operand is the register updated by the one-previous instruction (detected by tag2 == Src1 ), then we must use the value stored in the Forwarding Register FR2 as input operand for the ALU !!!
Otherwise, we can use the correct value of the source register from the "A"-register (see example click here)

The 2nd multiplexor that selects the second input of the ALU:

The 2nd multiplexor (colored yellow) will select from among: IR1 (contains the constant operand inside an instruction), the "B"-register, Forwarding Register FR1 and Forwarding Register FR2

The selection logic of the first multiplexor is as follows:

if ( Branch Instruction OR Imm bit == 1 (uses constant as operand) ) select register IR1 as output operand else if ( tag1 == Src1 ) select Forward Register FR1 as output operand else if ( tag2 == Src1 ) select Forward Register FR2 as output operand else select the "B"-register as output operand

The reason for this logic is because:

An Branch instruction will always use the constant value (offset)
An ALU/LD/STore instruction (e.g.: add r1,r2,#3) that uses a constant value will also use register IR1 (which contains the constant) as first operand.
Therefore, if the instruction is a branch instruction OR the Imm bit (= use constant operand) in the instruction is the == 1, the mux will output register IR1 for the ALU

If the instruction is an ALU, Load or Store isntruction (= a non-branch instruction), then: (the instruction can use values from a general purpose register !!!):

If the source register 1 operand is the register updated by the last instruction (detected by tag1 == Src1 ), then we must use the value stored in the Forwarding Register FR1 as input operand for the ALU !!!
Otherwise, if the source register 1 operand is the register updated by the one-previous instruction (detected by tag2 == Src1 ), then we must use the value stored in the Forwarding Register FR2 as input operand for the ALU !!!
Otherwise, we can use the correct value of the source register from the "B"-register (see example click here)

Let us first look at an example and convince ourselves that the solution works. Later, I will should you how the two multiplexor in the EX stage is constructed (it's pretty straightforward).

I will use the same example:
- Reconsider the following program that is executed by the IMPROVED pipeline:
```
   add r1, r2, r3            R2=192, R3=48, R4=1, R5=2, R6=3, R7=4
   add r4, r1, r4
   add r5, r1, r5
   add r6, r1, r6
   add r7, r1, r7
   ...
```
- The correct behavior (one that an assembler programmer would expect) is: R1 := R2+R3 = 20, then the other instructions will add R1=20 to registers R4 (20+1), R5 (20+8), R6 (20+0) and R7
Slideshow:

(Cycle 1: fetch ALU instruction in ID stage)

(Start of cycle 2: ID stage fetch operands)

(End of cycle 2: operand fetched, instruction advances)

(Start of cycle 3: EX stage computes on operands, ID stage fetches the old value of r1 )

(End of cycle 3: EX stage stored result in ALUo and FR1 registers, ID stage fetched old r1 value, all instructions advance)

(Start of cycle 4: MEM stage execute LD/ST/Branch instruction, EX stage selects FR1 value for r1, ID stage fetches old r1 value again )

(End of cycle 4: MEM stage did nothing, EX computed a correct value, ID stage fetched old r1 value (again), all instructions advance)

(Start of cylce 5: WB stage updates destination register r1, EX stage selects FR2 value for r1, ID stage fetches correct r1 value !! )

(End of cycle 5: destination register updated, instruction discarded)

❮ ❯
CPU Cycle 1
- At start of the CPU cycle, the IF stage sends out PC
- At end of the CPU cycle, the IR(ID) register is updated with the instruction fetched (add r1, r2, r3)
  
  State at the end of CPU cycle 1:
  - The picture above depicts the content of the CPU at end of the first CPU cycle (and the start of the 2nd cycle) - nothing special happened....
CPU Cycle 2
- At start of the CPU cycle, the ID stage sends out selection signal that selects values from R2 and R3
- At end of the CPU cycle, the "A" register is updated with R2 = 192, the "B" register is updated with R3= 48.
- Also, at the end of the CPU cycle, the instruction (add r1, r2, r3) is moved into IR(EX) and instruction "add r4, r1, r4" is fetched into IR(ID)
  State at the end of CPU cycle 2:
- The picture above depicts the content of the CPU at end of the second CPU cycle (and the start of the 3rd cycle) - nothing special has happened so far....
CPU Cycle 3
- At start of the CPU cycle, the EX stage selects values from R2 and R3 for the ALU, use the ALU opcode to make ALU add the input values forming the result 240 (which will become the value of R1)
  The difference now is the result (240) will also be written into the Forwarding Register FR1 along with the register tag = 001 (indicating register R1)
  Also, at start of the CPU cycle, the ID stage selects R4 and R1 to be copied into the "A" and "B" registers,
  Notice that an OLD value of R1 (= 12) will still be fetched into B. (That is not a problem because we will find a way to obtain the more recent value from the forwarding registers - see next CPU cycle).
  State at the end of CPU cycle #3:
- The picture above depicts the content of the CPU at end of the 3rd CPU cycle (and the start of the 4th cycle)
- Something special is about to happen:
CPU Cycle 4
- I will go a bit slower now to highlight the data forwarding technique
- At start of the CPU cycle, the EX stage will apply the new selection logic to select the operand ( click here )
  For the first operand, the value from the "A" register (R4) is selected.
  But for the second operand, the value of the Forwarding register FR1 is select -- because (tag1 (001) == Src2 (001)) !!!.
  So the ALU will add R4 with the new value 240 of R1 (which has not arrived to R1 yet !!!)
  State at the start of CPU cycle 4:
- At the end of the cycle, ALUo will be updated with 240+1 = 241, which is the CORRECT outcome because R1 will be equal to 240).
- Also at the end of the cycle, the value in the Forwarding Register FR1 is copied (preserved for one more CPU cycle) to Forwarding Register FR2.
- At the end of the cycle, the result of the ALU (241) is copied into Forwarding Register FR1 along with the register tag "100" (= binary 4) indicating register R4 - this Forwarding Register value will aid in "correcting" a subsequent instruction that use R4 as a source operand !!!
  State at the end of CPU cycle #4:
- The picture above depicts the content of the CPU at end of the 4th CPU cycle (and the start of the 5th cycle)
- Note in the above picture that the NEW value (240) of R1 is still available in Forwarding Register FR2 as one of the inputs of the Multiplexor (see the red line) along with the old value of R1 (see the magenta line).
- Now you should understand why we need two forwarding registers cascaded one after the other. The register value R1 is retained for exactly 2 CPU cycles, because we have previously determined that 2 instructions fails to obtain the more recent value.
CPU Cycle 5
- At start of the CPU cycle, the EX stage will apply the same new selection logic to select the operand ( click here )
  For the first operand, the value from the A register (R5) is selected.
  For the second operand, the value of the Forwarding register FR2 is selected -- because (tag2 (001) == Src2 (001)).
  So the ALU will add R5 with the new value (= 240) of R1 (which has still not arrived at R1 yet !!!)
  State at the start of CPU cycle #5:
- At the end of the cycle, ALUo will be updated with 240+2 = 242, which is the correct result because the new value of R1 is 240.
- Also at the end of the cycle, the value in the Forwarding Register FR1 is copied (preserved for one more CPU cycle) to Forwarding Register FR2 and the value of R1 will not be stored in the forwarding registers !!!
  NOTE: That is not a problem because we have previously determined that the 3rd instruction following "add r1,r2,r3" can obtain the correct value of R1 using the "A"/"B"-registers.
- At the end of the cycle, the result of the ALU (242) is copied into Forwarding Register FR1 along with the register tag "101" indicating register R5 - this Forwarding Register value will aid in "correcting" a subsequent instruction that use R5 as a source operand (while the value in Forwarding Register FR2 is used to correct instructions that use R4 as source operand) !!!
  State at the end of CPU cyle #5:
- The picture above depicts the content of the CPU at end of the 5th CPU cycle (and the start of the 6th cycle)

Implementing the multiplexors used in Data forwarding

The input to the ALU is now selected from many different sources.

The first input of the ALU can be one of the following:

The value of the PC - only when the instruction is a branch instruction
The "A"-register
The first frowarding register (ForwReg1)
The second frowarding register (ForwReg1)

The selection logic (algorithm) can be formulated as follows:

if ( instruction is a BRANCH instruction ) select PC1; else if ( Tag1 in ForwReg1 == Src1 in instruction ) select ForwReg1; // Because this reg. has the most recent value else if ( Tag2 in ForwReg2 == Src1 in instruction ) select ForwReg2; // Because this reg. has the next recent value else select A-register;

The selection algorithm is implemented in hardware with compare circuits and multiplexors.
(I call this a "multiplexor", but in reality, it is a hardware if-statement !!!)
The following circuit can be used to compare if 2 binary numbers are equal:
- The circuit will output ONE if and only if ALL pairs of bits are equal....
This equality circuit is used with a number of multiplexors to construct the "if-else" selection algorithm in hardware.

The following circuit diagram shows the implementation of the forwarding algorithm with multiplexors for the first input of the ALU (the yellow multiplexor):

This circuit (hardware) implements the following if-statement:

(The red circle represents the "compare equal" circuit above).

Notice that the last multiplexor will affect the input most, and this mux selects PC1 for the branch and the value from the second last multiplexor.
The PC1 value is selected when the instruction is a BRANCH instruction and otherwise, the value from the second last multiplexor is selected.

The second multiplexor is constructed in a similar manner and is left out for brevity

Example Program: (Demo above code)

/home/cs355001/demo/pipeline/4a-ALU-hazard-sol Executes: 10 12 // mov r1,#12 18 192 // mov r2,#192 26 48 // mov r3,#48 34 1 // mov r4,#1 42 2 // mov r5,#2 50 3 // mov r6,#3 58 4 // mov r7,#4 0 0 // nop 0 0 // nop 0 0 // nop 0 0 // nop 8 19 // add r1,r2,r3 (R1=R2+R3) 32 33 // add r4,r1,r4 (R4 = R1 + R4) (R1 forwarded) 40 41 // add r5,r1,r5 (R5 = R1 + R5) (R1 forwarded) 48 49 // add r6,r1,r6 (R6 = R1 + R6) 56 57 // add r7,r1,r7 (R7 = R1 + R7)