Handling "≥"-constraints: the two-phase Simplex method

An example of Linear Program with "≥"-constraints

Consider the following Linear Program (constrained optimilization problem with linear objective and constraints):

max: z = x₁ + x₂ s.t.: x₁ + 2x₂ ≤ 8 3x₁ + 2x₂ ≤ 12 x₁ + 3x₂ ≥ 3 <----- **** New "feature ****

The feasibility region:

Converting "≥"-constraints into "="-constraints

Fact:

Example:

Constraint: x₁ + x₂ ≥ 5 (x₁ ≥ 0, x₂ ≥ 0 are implied)
Equivalent constraint: x₁ + x₂ - s₁ = 5 (x₁ ≥ 0, x₂ ≥ 0 and s₁ ≥ 0 are implied) Notice: negative sign (-s₁)

The effect of "≥"-constraints on the initial Simplex Tableau

Linear Program:

max: z = x₁ + x₂ s.t.: x₁ + 2x₂ ≤ 8 3x₁ + 2x₂ ≤ 12 x₁ + 3x₂ ≥ 3

Re-write the Linear Programming problem as a system of linear equations (=):

z = x₁ + x₂ ⇒ z - x₁ - x₂ = 0 x₁ + 2x₂ ≤ 8 ⇒ x₁ + 2x₂ + s₁ = 8 3x₁ + 2x₂ ≤ 12 ⇒ 3x₁ + 2x₂ + s₂ = 12 x₁ + 3x₂ ≥ 3 ⇒ x₁ + 3x₂ - s₃ = 3

Put the linear system into a matrix:

Linear system: z - x₁ - x₂ = 0 x₁ + 2x₂ + s₁ = 8 3x₁ + 2x₂ + s₂ = 12 x₁ + 3x₂ - s₃ = 3
Lined up: -x₁ - x₂ + z = 0 x₁ + 2x₂ + s₁ = 8 3x₁ + 2x₂ + s₂ = 12 x₁ + 3x₂ - s₃ = 3
Matrix form: x₁ x₂ s₁ s₂ s₃ z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3

Note:

We may detect the basis easier if you write the above matrix as follows:

x₁ x₂ s₁ s₂ s₃ z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 1 0 | -3

The basis variables are:

The corresponding "basic" solution is: (I put "basic" in quotes because it is not a basic solution !!!)

x₁ = 0 (non-basic variable = 0))
x₂ = 0 (non-basic variable = 0))
s₁ = 8 (basic variable = RHS))
s₂ = 12 (basic variable = RHS))
s₃ = −3 (basic variable = RHS)) ******** Trouble !!! ******
z = 0 (basic variable = RHS))

Big trouble:

s₃ = −3 violates the (implied) constraint:
This solution is not feasible !!!
This "basic" solution is not basic at all --- because a basic solution must be feasible

Techniques to find an initial (feasible) basic solution
- There are two (2) techniques that you can use to find a (feasible) basic solution when the Linear Program contains "≥"-constraint:
  I like the two-phase Simplex method because it's more intuitive (so I'll teach you that :-))

Artificial variables: a trick used in Simplex Method to obtain a feasible basic solution

Simplex Tableau:

x₁ x₂ s₁ s₂ s₃ z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3

Trick (technique) to obtain an initial basic solution:

For every "≥"-constraint, add an artificial variable with a canonical unit basic vector

Example:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ | | | artificial variable | (with canonical unit vector) | The ≥-constraint

Purpose of artificial variables:

The artificial variables (such as a₁ above) are not part of the optimization problem

In other words:

We cannot apply the Simplex Method as long as the Simplex Tableau contains artificial variables
(On the other hand, surplus variables (introduced by "≤"-constraints are OK))

The artificial variables (like a₁) are used temporarilly to help us find (compute) a feasible basic solution

Problem:

Removing artificial variables

Our Simplex tableau so far is as follows:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3

The corresponding solution is:

z = 0 s₁ = 8 s₂ = 12 a₁ = 3 <--- "Remove" this variable

How to "remove" a variables from an equation:

Therefore:

We must find a solution (i.e., give values to all variables in the Simplex tableau) that has the property that:

Finding a solution with all artificial variable = 0

Our Simplex tableau so far is as follows:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 <-- Objective function 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3

Fact:

Only the constraints determine the feasibility of a solution
In other words:

Trick to obtain a solution where all artificial variables are equal to 0:

Construct a LP using the same set of constraints but with a different objective function where:
(Because All artificial variables, the artificial variables now has no effect on the equations and we can "remove" them !!!)

An objective function where the optimal solution is found using:

All artificial variables = 0

is:

max: -a₁ - a₂ - ... - a_m (Remember that: a₁ ≥ 0 a₂ ≥ 0 ... a_m ≥ 0 )

Because:

-a₁ - a₂ - ... - a_m ≤ 0 (is always negative) since: a₁ ≥ 0 a₂ ≥ 0 ... a_m ≥ 0 )
The maximum value is therefore: 0

And the only way to obtain this maximum is:

a₁ = 0 a₂ = 0 ... a_m = 0

Now:

We solve the Linear Program with:
When the Simplex Method terminates, we will have:

More importantly:

Because the constraints used are the same as those in our original Linear Program (and a feasible solution is only determined by the constraints used):

The solution that we obtained by solving a different objective function:
is also a feasible solution of the original Linear Program !!!
(Because they use the same set of constraints !!!)

Terminology:

Phase 1 of the Simplex method:

Overview of the two-phased Simplex Method

The 2-phased Simplex Method:

1. Set up an initial Simplex Tableau: 1a. Add a surplus variable s_i to each: "≤"-constraint and "≥"-constraint Note: surplus variables of "≥"-constraint has negative sign ! 2. If there is one or more "≥"-constraints then: 2a: Add an artificial variable a_i to each ≥ constraint 2b. Use the Simplex method to solve: max: -a₁ - a₂ - ... - a_n Using same set of constraints Note: you need to fix the Simplex Tableau first (see example) 2c. When Simplex method terminates, replace the objective row of the Final Simplex Tableau by the original objective function 3. Use the Simplex method to solve the LP Note: you need to fix the Simplex Tableau first (see example)

Note:

Example: 2-phased Simplex Method

Consider the LP with an "≥"-constraint:

max: z = x₁ + x₂ s.t.: x₁ + 2x₂ ≤ 8 3x₁ + 2x₂ ≤ 12 x₁ + 3x₂ ≥ 3

Step 1: Add a surplus variable to each "≤"- and "≥"-constraint:

max: z - x₁ - x₂ = 0 s.t.: x₁ + 2x₂ + s₁ = 8 3x₁ + 2x₂ + s₂ = 12 x₁ + 3x₂ + s₃ = 3

Re-written in table form:

x₁ x₂ s₁ s₂ s₃ z | RHS ===============================+==== -1 -1 0 0 0 1 | 0 1 2 1 0 0 0 | 8 3 2 0 1 0 0 | 12 1 3 0 0 -1 0 | 3

(which did not have a feasible initial basic solution)

Step 2: (this step is necessary because there is one "≥"-constraint)

Add an artificial variable to each "≥"-constraint:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -1 0 0 0 0 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ | |

Remove the artificial variables by solving:

max: z = -a₁ Or: max: z + a₁

using the same set of constraints.

The Linear Program that we must solve is:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 (Objective function = z + a₁) 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3

Before we can solve this Linear Program:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 1 2 1 0 0 0 0 | 8 3 2 0 1 0 0 0 | 12 1 3 0 0 -1 1 0 | 3 ^ ^ ^ | | | We onloy have 3 canonical unit vectors !!!

We must "fix" it (because there are 4 rows and only 3 canonical unit vectors) by:

Making all artificial variables into basic variables

Before fixing:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== 0 0 0 0 0 1 1 | 0 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) 3 2 0 1 0 0 0 | 12 (row 3) 1 3 0 0 -1 1 0 | 3 (row 4)

Make a₁'s column into a unit vector by subtracting row 4 from row 1:

x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) 3 2 0 1 0 0 0 | 12 (row 3) 1 3 0 0 -1 1 0 | 3 (row 4)

Now we have 4 canonical unit vector and we can apply the Simplex Method !

Run the Simplex Method:

Iteration 1:

largest neg coef | V x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) 1 2 1 0 0 0 0 | 8 (row 2) (test ratio = 4) 3 2 0 1 0 0 0 | 12 (row 3) (test ratio = 6) 1 3 0 0 -1 1 0 | 3 (row 4) (test ratio = 1)
Normalize pivot row: x₁ x₂ s₁ s₂ s₃ a₁ z | RHS =====================================+==== -1 -3 0 0 1 0 1 | -3 (row 1) + 3 x row 4 1 2 1 0 0 0 0 | 8 (row 2) - 2 x row 4 3 2 0 1 0 0 0 | 12 (row 3) - 2 x row 4 0.3 1 0 0 -0.3 0.3 0 | 1 (row 4)
Sweep the pivot column: x₁ x₂ s₁ s₂ s₃ a₁ z | RHS ==========================================+==== 0.0 0.0 0.0 0.0 0.0 1.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 -0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 -0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0.3 0 | 1.0

Iteration 2:
Done.

Basic solution:

x₁ = 0 x₂ = 1 s₁ = 6 s₂ = 10 s₃ = 0 a₁ = 0

Because a₁ = 0, we can now remove this variable without affecting the values of the other variables:

x₁ x₂ s₁ s₂ s₃ z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0

Notice that the baisc solution:

x₁ = 0 x₂ = 1 s₁ = 6 s₂ = 10 s₃ = 0 ~~a₁ = 0~~

satisfy all original constraint (so it is a feasible solution for the original problem)

Step 3: Use the Simplex method (and the found feasible solution) to solve the original LP

The Simplex Tableau:

x₁ x₂ s₁ s₂ s₃ z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 (Objective: max -a₁) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0

represents the following feasible solution:

x₁ = 0 x₂ = 1 s₁ = 6 s₂ = 10 s₃ = 0

(These values satisfies all constraints !!!)

We now use this feasible solution to solve the original objective function

This is done by:

Keeping all the constraint rows

Replace the objective function by the original objective function:

max: z = x₁ + x₂ or: max: z - x₁ - x₂ = 0

In other words:

Our Simplex Tableau: x₁ x₂ s₁ s₂ s₃ z | RHS =====================================+========= 0.0 0.0 0.0 0.0 0.0 1 | 0.0 (Objective: max z+a₁) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | This was a unit vector
New Simplex Tableau: x₁ x₂ s₁ s₂ s₃ z | RHS =======================================+==== -1.0 -1.0 0.0 0.0 0.0 1 | 0.0 (Objective: max z-s₁-s₂) 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 <--- Keep constraints 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | It is no longer a unit vector: fix it first !

Preparation for Simplex: fix all corrupted unit vectors

x₁ x₂ s₁ s₂ s₃ z | RHS =======================================+==== -1.0 -1.0 0.0 0.0 0.0 1 | 0.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0 ^ | Corrupted unit vector

Fix it: (make the column into a unit vector again) by adding row 4 to row 1:

x₁ x₂ s₁ s₂ s₃ z | RHS =======================================+==== -0.7 0.0 0.0 0.0 -0.3 1 | 1.0 0.3 0.0 1.0 0.0 0.7 0 | 6.0 2.3 0.0 0.0 1.0 0.7 0 | 10.0 0.3 1.0 0.0 0.0 -0.3 0 | 1.0

Note:

Corresponding basic solution:
which is a corner point in the feasible region:

Now we can run the Simplex Method to solve the original problem:

Starting Simplex Tableau:

Current basic solution: x₁ = 0, x₂ = 1, (z = 1)

After pivot operation:

Current basic solution: x₁ = 3, x₂ = 0, (z = 3)

After pivot operation:

Current basic solution: x₁ = 4, x₂ = 0, (z = 4)

After pivot step:

x₁ x₂ s₁ s₂ s₃ z | RHS =======================================+==== 0.0 0.0 0.3 0.2 0.0 1 | 5.0 0.0 1.0 0.7 -0.2 0.0 0 | 3.0 0.0 0.0 1.7 -0.2 1.0 0 | 8.0 1.0 0.0 -0.5 0.5 0.0 0 | 2.0 No improving pivots left ===> optimal !

Corresponding solution: (x₁ = 2, x₂ = 3, z = 5)