Review:   Program

  • Program = Algorithm + Data Structure

  • Algorithm:

    • Algorithm = a method for solving problems expressed as a sequence of steps suitable for execution by a computer

    Algorithms takes time to run....

  • Data structure:

    • Data structure = variables storing information used by an algorithm that is organized in a specific way

    Data structure take up memory space...

  • A "good" program should ideally be one that:

    • Runs fast...                  and
    • Uses very little memory...

How to measure the "goodness" of computer programs

  • Question:

    • How can do we measure the "goodness" of a program/algorithm ?

  • Possible performance mesaures:

    • Running time (the shorter, the better)

    • Memory utilization (the less, the better)

    • Amount of code ??

    • Etc

  • Most commonly used performance (= "goodness") measure:

    • Running time

How to measure the running time of a program/algorithm ?

  • Stop watch:  00:00:00 

    Problems will real time as measure:

    • The same program can have different running time of different computers...

    • Different inputs can result in different running time - hard to find their relationship...

  • An objective measure:

    • Count the number of instructions executed by a program for a given input size

    This is not practical....

  • A more practical measure:

    • Count the number of "primitive operations" executed by a program for a given input size

What is a primitive operation ?

  • Recall that algorithms make repeated steps towards the solution...

  • The primitive operation is a step in the algorithm

  • Example:

    • Suppose the main body of an algorithm looks like this:

         for ( int i = 0; i < N; i++ }
         {
            S1;   --+
            S2;     |
            ...     | 1 step = execute S1, S2, ..., SN
            SN;   --+
         }

    • Then the primitive operation of this algorithm consists of the statements S1; S2; ...; SN;

Principles of Algorithm Analysis

  • Algorithm Analysis consists of:

    1. Determine frequency (= count) of primitive operations

    2. Characterize the frequency as a function of the input size

  • The Algorithm Analysis must:

    • Take into account all possible inputs ("good" one and "bad" ones)

    • Be independent of hardware/software environment

    • Be independent from the programming language

    • Give a good estimate that is proportional to the actual running time of the algorithm

"Good" inputs, "Bad" inputs and "Average" case

  • Input data can affect the running time of algorithms !!!

  • An algorithm may run:

    • Faster with some "good" input
    • Slower with some "bad" input

    Example: Linear Search in an array

    • Is fast if the first element in array was the item you were looking for

    • Is slow if the last element in array was the item you were looking for

  • There are 3 (actually 2...) cases to consider when doing algorithm analysis:

    • The best case - not studied because you can't count on luck...
    • The worst cast - gives an upper bound
    • The "average" case - what you can expect...

Worst-case vs. Average-case Analysis

  • Worst case analysis:

    • Provides an upper bound on the running time of an algorithm

    • The analysis is easier to do (compared to an average case analysis)

  • Average case analysis:

    • Take the average (running time) over all possible inputs of the same size

    • The analysis depends on input distribution

    • The analysis is harder to do because it uses probability techniques

Techniques used in Algorithm Analysis

  • There are 2 main techniques used in Algorithm Analysis:

    1. Loop Analysis

    2. Recurrence relations

  • A program spends the most amount of time in loops

    One of the technique used in Algorithm Analysis is loop analysis:

    • Loop Analysis = count the number of times that a loop body is executed

  • Some algorithm is recursive:

    • The run time complexity of recursive algorithms are often expression as a recurrence relation

      E.g.: C(n) = 2×C(n/2) + 1

    The other technique is solving recurence relations.