Posix Thread (pthread) Programming

Introduction to Multi-threaded (Parallel) Programming

Multi-processors
- Recall that a Shared Memory multi-processor has the following (simplified) architecture:
  
  (NOTE: the inter-connection network is irrelevant to the programming and has been omitted).
  - The memory is shared by all CPUs
  - Each CPU fetches its own instruction and executes it

Processes and Threads

A process (in any operating system, e.g. UNIX) is created whenever a program is run

Different programs/processes can be executed simultaneously by a multi-processor computer (computer with multiple CPUs):

In the figure, the instructions at the arrow in both processes may be executed simultaneously (assuming we have parallel hardware)
NOTE: computers execute assembler instructions, not high level language statements, such as "i = i + 1". Every programming language statement (like "i = i + 1"), is compiled into multiple assembler instructions.
This detail will be improtant when we have multi-threaded execution of the same program.

A thread of execution (or thread for short) is a location in the program where a processor (CPU) is fetching and executing instructions
In other words: a thread makes progress in executing a program

Each process has at least one thread of execution.
Most (if not all) programs that you have seen probably have ONE SINGLE thread of execution
These are called SINGLE threaded programs
We will soon see MULTI threaded programs

Summary of Properties of processes:
- Different processes Do not share instruction (program) code
- Different processes Do not share variables
- Different processes may share a minimum amount of information that allow them to communicate with each other through the operating system kernel using a system call (i.e., with the aid of the Operating System)
  - E.g., UNIX processes share file descriptors which them to create pipes to communicate with each other.
- In UNIX, whenever processes need to communicate with one another, they must invoke some system call.
Multi-threaded programs
- A thread can be thought as a flow of control or line of execution inside a process
  There is at least one thread within every process (otherwise, the process will not run :-))
- "Traditional" processes are single threaded
  A single threaded process has exactly one thread...
- A multi-threaded program has more than one thread
- So a multi-threaded program has multiple flows of control
  In other words:
  - If the computer is a multi-processor, the computer can be simultaneously executing at multiple places inside the SAME program !!!
- Pictorially:
- In the above picture, the computer is executing two instructions from process 1 and three instructions from process 2 simultaneously (that is assuming we DO have sufficient parallel hardware - in this case, at least 5 CPUs !!!)
Creating new threads
- Creating a new thread:
  Effect:
- Example:

Example Multi-Threaded Program

Just to get you exicted on writting multi-threaded programs (later, I'll get you acquainted with the caveats of parallel program execution), here is a very simple example of a multi-threaded program

Example Multi-threaded program:

#include <pthread.h> pthread_t tid[100]; // Each thread executes the following function: void *worker(void *arg) { int i; i = pthread_self(); cout << "Hello, I am thread # " << i << endl; return(NULL); /* Thread exits (dies) */ } /* ======================= MAIN ======================= */ int main(int argc, char *argv[]) { int i, num_threads; num_threads = atoi(argv[1]); /* ------ Create threads ------ */ for (i = 0; i < num_threads; i = i + 1) { if ( pthread_create(&tid[i], NULL, worker, NULL) ) { cout << "Cannot create thread" << endl; exit(1); } } // Wait for all threads to terminate for (i = 0; i < num_threads; i = i + 1) pthread_join(tid[i], NULL); exit(0); }

The above program makes use of two functions from the PThreads API
1. pthread_create(&tid[i], NULL, worker, NULL) will:
  - Create a new thread (new beginning of execution)
  - Tell the new thread to run the function worker (well, the thread must execute some computer instruction and it needs NOT start in the main() function - that would be too restrictive)
  - Each created thread has a unique thread ID. The thread ID of the new thread will be stored in the variable tid[i] (the first parameter of the function call - notice the address of tid[i] is passed, so the pthread_create(&tid[i], NULL, worker, NULL) can update the variable !)
2. pthread_join(tid[i], NULL, worker, NULL) will pause the running thread until the target thread with thread ID tid[i] completes (the thread completes when the function worker returns).
Example Program: (Demo above code)
- Prog file: click here
Compile with: g++ -pthread thread01.C (linux)
Run the program several times and you will notice that the threads prints things out in different order...
We do not have control on when each thread is run....

The big deal about multi-threaded programs

OK, so many different parts of a multi-threaded program can be executed by the multi-processor computer --- it will run some multiple times faster... right ?....
WRONG...
Huh ? Why not ? What's the problem ?

Illustrative Example Program: Consider the following program where each thread adds 10,000 to the variable N, but the addition is one 1 at a time:

#include <pthread.h> int N; pthread_t tid[100]; // Each thread executes the following function: void *worker(void *arg) { int i, k, s; for (i = 0; i < 10000; i = i + 1) { N = N + 1; } cout << "Added 10000 to N" << endl; return(NULL); /* Thread exits (dies) */ } /* ======================= MAIN ======================= */ int main(int argc, char *argv[]) { int i, num_threads; num_threads = atoi(argv[1]); /* ------ Create threads ------ */ for (i = 0; i < num_threads; i = i + 1) { if ( pthread_create(&tid[i], NULL, worker, NULL) ) { cout << "Cannot create thread" << endl; exit(1); } } N = 0; // Wait for all threads to terminate for (i = 0; i < num_threads; i = i + 1) pthread_join(tid[i], NULL); cout << "N = " << N << endl << endl; exit(0); }

Example Program: (Caveat of multi-threaded programs) --- click here
- Compile with: g++ -pthread thread02.C (on Linux)
- Run to see some interesting results....

Properties of threads:
- Threads are flows of control within the SAME program
- As such, all threads share the same memory space !!!
  - All threads share all global variables in the program
  - Threads do not share their local variables (unless they convey the location (address) of their variable to other threads)
- Threads can be ready to run or blocked.
  Threads that are ready to run will be executed in no particular order by the computer system (the programmer does not have (nor need) control over the therad scheduling --- you could do some thead scheduling, but it is not worth the effort)
- NOTE: Because the global variables are shared by all threads, these types of variables can be used by threads to communicate with one another --- through the writing and reading of global variables
  Since variables are stored in memory, this method of communication is also known as shared memory

How can we explain the results observed in thread02.C with the properties of threads ?

Recall that computers execute assembler instructions
Recall that the memory stores operands and the CPU performs the operations (such as +)
The statement "N = N + 1" is translated into multiple assembler instructions:
1. Instruction 1: Read (get) the value from variable N in memory into the CPU
2. Instruction 2: Add 1 from this value
3. Instruction 3: Write (put) the result from the CPU in variable N in memory
Remember that in a multi-processor (such as "compute.mathcs.emory.edu"), all processors work simultaneously.
When multiple processors execute the statement "N = N + 1", incorrect outcome can result due to interspersed execution

Example of interspersed execution of 2 threads that result in a "missed update":

Thread 1 on Thread 2 on Memory CPU 1 CPU 2 ============== =================== ================= N = 1234 Read N --> 1234 Add 1 --> 1235 Read N --> 1234 N = 1235 Write N Add 1 --> 1235 N = 1235 Write N

The second value N = 1235 should have been N = 1236" if execution of "N = N + 1" was not interspersed !!!

Parallel programs: some lessons learned
- Multi-threaded programs do NOT behaved like single threaded programs where only one thing happens at a time.
- Due to concurrent execution unexpected behavior may arise
- Parallel program are very hard to write and especially hard to DEBUG (because the behavior is not repeatable - different executions can produce different outcomes)
- Always try to limited variable sharing in parallel programming - the more variables are shared, the more opportunities for problems (but variables sharing is inevitable in parallel programs)

Share and non-shared variables

When you write multi-threaded (parallel) programs, you have to understand the difference between SHARED and NON-SHARED variables
A thread executes some function...
The scoping rules that we learned in this webpage (click here) HAS NOT CHANGED
- If a variable is accessible under single threaded programming, the variable is also accessible under multi-threaded programming.
- If a variable is NOT accessible under single threaded programming, the variable is also NOT accessible under multi-threaded programming.
Therefore:
- All GLOBAL variables are shared (accessible) by all threads
- All LOCAL variables (defined in functions) are NOT shared (NOT accessible) by threads other than the one that executes that function

Now you should understand the difference between the behavior of the following 2 functions when they are executed concurrently by multiple threads:

void *worker(void *arg) { int N; int i; // Local (non-shared) variable N = 0; for (i = 0; i < 10000; i = i + 1) { N = N + 1; } cout << N; return(NULL); /* Thread exits (dies) */ }

int i; // Global (SHARED !!!) variable void *worker(void *arg) { int N; N = 0; for (i = 0; i < 10000; i = i + 1) { N = N + 1; } cout << N; return(NULL); /* Thread exits (dies) */ }

The first program (yellow background) prints N = 10000 by all threads... the variable i kept an accurate count within the loop --- because each thread has its own copy of the variable i
The second program (lightblue background) prints unpredictable values for N by all threads... the variable i kept an INACCURATE count within the loop --- because the variable i was SHARED by many different threads

Example Program: (shared vs non-shared variables) ---
- Threads using non-shared i: click here
- Threads using shared i: click here
- Compile with: g++ -pthread thread03.C and g++ -pthread thread03b.C (on Linux)