Recovery using a redo log

Slideshow:

How to perform recovery using nonquiecent checkpointing using REDO logging

After a system failure, the REDO log file can be in one of 2 possible states:

When scanning the REDO log file backwards, we find a <END CKPT> record first

How to perform recovery using nonquiecent checkpointing using REDO logging

After a system failure, the REDO log file can be in one of 2 possible states:

When scanning the REDO log file backwards, we find a <START CKPT> record first

Recall: how to recovery using a REDO log

We need to search different portion in the REDO log for these different case....

Regardless: we must find all committed transactions (in the REDO log file)

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario:

I.e.: the last log checkpoint operation has completed successfull

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario: we know that:

Because the <END CKPT> is only written after the all changes made by committed transactions have been written to disk !

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario: we can determine cut off point in REDO log file:

Because these transactions have written their data to disk (don't need to REDO them)

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario: find remaining committed transaction:

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario: portion of the REDO log used in recovery:

REDO logging recovery procedure for case 1 (completed checkpoint)

Case 1 scenario: how to recover using a REDO log:

REDO logging recovery procedure for case 1 (completed checkpoint)

Pseudo code: how to recover using a REDO log:

REDO logging recovery procedure for case 1 - Example recovery

Example recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example recovery

Example recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example recovery

Example recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example recovery

Example recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example recovery

Example recovery for case 1 scenario:

Result: we will redo the updates of the (committed) transactions T₂ and T₃

REDO logging recovery procedure for case 1 - Example 2

Example 2 recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example 2

Example 2 recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example 2

Example 2 recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example 2

Example 2 recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example 2

Example 2 recovery for case 1 scenario:

REDO logging recovery procedure for case 1 - Example 2

Observation:

REDO logging recovery procedure for case 2 (crashed during checkpointing)

Case 2 scenario:

System has crashed during a checkpointing operation...

REDO logging recovery procedure for case 2 (crashed during checkpointing)

Case 2 scenario: the worst case scenario:

In order to make sure that all updates from committed transactions are incorporated, we must assume the worst case scenario for the checkpoint operation....

REDO logging recovery procedure for case 2 (crashed during checkpointing)

Case 2 scenario: the worst case scenario:

REDO logging recovery procedure for case 2 (crashed during checkpointing)

Case 2 scenario: what to do ?

REDO logging recovery procedure for case 2 - Example recovery

Example recovery for case 2 scenario:

REDO logging recovery procedure for case 2 - Example recovery

Example recovery for case 2 scenario:

Conclussion: we must go further back and look for a <END CKPT> record

REDO logging recovery procedure for case 2 - Example recovery

Example recovery for case 2 scenario:

Recovery: redo all committed transactions

REDO logging recovery procedure for case 2 - Example recovery

Example recovery for case 2 scenario:

Recovering using nonquiescent checkpointing

When scanning the log file backwards, you can find one of the following:

You find a <END CKPT> record first:
This is the case when:
You find a <START CKPT(T₁, T₂, ..., T_k)> record first:
This is the case when:

Recovery algorithm in redo logging
- Recall: in redo logging
- Difference in crash scenarios of case 1 and case 2:

Recovering in case 1

Where to find the necessary redo log records in case 1:

Given that we find a <END CKPT> record first:
By the checkpointing algorithm:
Therefore, in the recovery, we do not need to redo the changes made by committed transactions in this portion of the redo log:
We (still) have to redo the updates made by transactions that are committed after the <START CKPT> record:
The portion of the redo log that we need to search for find (= obtain) the redo log records is:

Summary: how to recover using redo logging in case 1:

Find the earliest <START T_i> log record where T_i is in the check point transaction list
Redo the updates made by the transactions that have committed after the <START CKPT> log record
Graphically:

Recovery Algorithm of redo logging for case 1:

/* ================================================== Step 1: identify the commited transactions ================================================== */ Scan the redo log backwards until the <START CKPT(T₁, T₂, ..., T_k) record: { identify the committed identify the uncommitted/aborted transactions } /* ================================================== Step 2: redo the commited transactions ================================================== */ Scan the redo log backwards until we found all <START T₁>, <START T₂>, ..., <START T_k> records Starting from this point, scan the redo log forewards { For ( each < T, A, v > ) { if ( T is commited ) { Update A with the (after) value v; // Redo the action !!! } } } /* ========================================================= Step 3: mark the uncommited tranasactions as failed.... ========================================================= */ For ( each T that is uncommited ) do { Write <ABORT T> to log; } Flush-Log

Examples of recovery using redo logging in case 1

Example 1:

Suppose the system crashes after writing both <COMMIT> records:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

We scan backwards and finds the <END CKPT> record first:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

We scan backwards upto <START CKPT(T₂)> to find all the commit records:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

Result:

Need to redo transactions T₂ and T₃

We scan backwards to find the <START T_x> records of all the commit transactions (T₂, T₃):

Redo log: <START T₁> <T₁, A, 5> <START T₂> <---------- Here !!!!! <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

From the line marked by Here, scan forward and redo all actions of T₂ and T₃:

Redo log: <START T₁> <T₁, A, 5> <START T₂> ******** Here !!!!! <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

Note: (log record pointers)

Example 2: (losing a completed (but uncommitted) transaction)

Suppose the system crashes after writing only one <COMMIT> record:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done +++++++++++++++++++++++++++++++++++++++ System crashed.... ~~<COMMIT T₃>~~ ===== T₃ is not comitted !!! <------------- ****** (OUTPUT for T₂ and T₃ happens later)

We scan backwards and finds the <END CKPT> record first:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done +++++++++++++++++++++++++++++++++++++++ System crashed.... ~~<COMMIT T₃>~~ ===== T₃ done (OUTPUT for T₂ and T₃ happens later)

We scan backwards upto <START CKPT(T₂)> to find all the commit records:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done +++++++++++++++++++++++++++++++++++++++ System crashed.... ~~<COMMIT T₃>~~ ===== T₃ done (OUTPUT for T₂ and T₃ happens later)

Result:

Need to redo transactions T₂ (only !!!)

We scan backwards to find the <START T_x> records of all the commit transactions (T₂):

Redo log: <START T₁> <T₁, A, 5> <START T₂> <------------- Here !!!!! <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done +++++++++++++++++++++++++++++++++++++++ System crashed.... ~~<COMMIT T₃>~~ ===== T₃ done (OUTPUT for T₂ and T₃ happens later)

From the line marked by Here, scan forward and redo all actions of T₂ :

Redo log: <START T₁> <T₁, A, 5> <START T₂> ******** Here !!!!! <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ <END CKPT> // Flush Log <COMMIT T₂> ===== T₂ done <COMMIT T₃> ===== T₃ done +++++++++++++++++++++++++++++++++++++++ System crashed.... (OUTPUT for T₂ and T₃ happens later)

Note:

Changes made by T₃ are not redone !!!

Finally, we write an abort record for T₃:

Note:

A "finished" transaction (T₃) can be voided if its <COMMIT> record is not written to the redo log !!!
To minimize this losing "committed" transaction phenomenon:

Recovering from case 2

Recovery algorithm for case 2:

We find a <START CKPT(T₁, T₂, ..., T_k)> record first:
During the checkpointing:
Without the <END CKPT> record:
Worst case scenario:

In the worst case scenario:

We can not obtain more information from the <START CKPT> log record !!!
It's like: not having the <START CKPT> record at all:

All we can do is go further back into the redo log:
and use the recovery procedure for case 1 !!!

Examples of recovery using redo logging in case 2

Example:

Suppose the system crashes before the check point ends:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ +++++++++++++++++++++++++++++++++++++++ System crashed.... ~~<END CKPT>~~ // Flush Log ~~<COMMIT T₂>~~ ===== T₂ done ~~<COMMIT T₃>~~ ===== T₃ done (OUTPUT for T₂ and T₃ happens later)

We scan backwards and finds the <START CKPT> record first:

Redo log: <START T₁> <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> ******** <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ +++++++++++++++++++++++++++++++++++++++ System crashed....

Search back to last <END CKPT> record (or to the start of the log file):

Redo log: <START T₁> <--------- HERE.... <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> ******** <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ +++++++++++++++++++++++++++++++++++++++ System crashed....

All commited transactions are:

T₁

Recovery procedure:

Redo the changes made by transactions T₁

Graphically:

Redo log: <START T₁> <--------- HERE.... <T₁, A, 5> <START T₂> <COMMIT T₁> ==== T₁ done <T₂, B, 10> ============================ <START CKPT(T₂)> ******** <T₂, C, 15> <--+ Between here: <START T₃> | **** OUTPUT(A) <T₃, D, 20> <--+ +++++++++++++++++++++++++++++++++++++++ System crashed....

Comment:

This action is correct, because the OUTPUT(A) operation of T₁ may not have been executed !!!

Log record pointers

Recall the recovery algorithm for an redo log:

/* ================================================== Step 1: identify the commited transactions ================================================== */ Scan the redo log backwards until first <START CKPT(T₁, T₂, ..., T_k) record: { identify the committed identify the uncommitted/aborted transactions } /* ================================================== Step 2: redo the commited transactions ================================================== */ Scan the redo log backwards until we found all <START T₁>, <START T₂>, ..., <START T_k> records Starting from this point, scan the redo log forewards { For ( each < T, A, v > where T is commited ) do { Update A with the (after) value v; // Redo the action !!! } } /* ========================================================= Step 3: mark the uncommited tranasactions as failed.... ========================================================= */ For ( each T that is uncommited ) do { Write <ABORT T> to log; } Flush-Log

We can find all <START T₁>, <START T₂>, ..., <START T_k> records
Finding the <START ..> log records can be expedited if we include the position of all <START T₁>, <START T₂>, ..., <START T_k> records in the <START CKPT> record: