---

Introduction to MPI Programming

---

• The Message Passing Interface (MPI) Standard
  • The MPI Standard is a message passing library standard based on the consensus of the MPI Forum

  • The MPI Forum consists over 40 organizations including vendors, researchers, developers and users.

  • The goal of MPI is to provide a portable, efficient and flexible standard for message passing parallel programming

    ---

  • Software download:

    Here is the Open MPI main website: click here You can download a free Open MPI (v 1.4) from this link click here The full MPI documentation can be found hre: click here

  ---

  ---

• Overview of the MPI API
  • The MPI header file

    Every C/C++ MPI program must include the MPI header file (which contains the MPI function type declarations)

    #include "mpi.h"

    ---

  • Every MPI function in C/C++ has the following prototype:

    int MPI_Xxxxx( param1, .... );

    Every MPI function returns an integer ERROR CODE

  • If an MPI function is successful (no error), it will return the error code MPI_SUCCESS (it's a symbolic name for some constant that you don't need to remember by value....)

  ---

  ---

• Structure of a MPI Program
  • Every MPI program has the following structure:

    #include "mpi.h" int main( int argc, char *argv[] ) { MPI_Init( &argc, &argv ); ..... ..... MPI_Finalize(); }

    ---

  • According the MPI documentation, the statement:

    MPI_Init( &argc, &argv );

    must be the first statement of the program

  • Most MPI implementations will relax the requirement to something like:
    It must be the first MPI function statement of the program

    In other words: you can write C/C++ statements before this, but to make sure your MPI program will run in future MPI implementation, it's a good idea to follow the standard....

  ---

  ---

• Hello World in MPI: version 1
  • Consider the following Hello World program in MPI:

    #include "mpi.h" #include <stdio.h> int main(int argc, char **argv) { MPI_Init( &argc, & argv ); printf("Hello World !\n"); MPI_Finalize(); }

    ---

  • Compile and run with: (only works if you have MPI installed :-))

    mpicc -o Hello Hello.c mpirun -np 4 ./Hello

    Output:

    $ mpirun -np 4 ./Hello Hello World ! Hello World ! Hello World ! Hello World !

    The message Hello World !" is printed 4 times, because the program is run on 4 processors (using the option -np 4)

  ---

  ---

  ---

  ---

• MPI Communicator Groups
  • Processes in MPI are divided into Communicator Groups

    A Communicator Group is a list/group of MPI processes that can communicate with one another

  ---

• The "default" MPI Communicator Groups: MPI_COMM_WORLD
  • There is a most commonly used Communicator Groups defined in MPI which is given the symbolic name MPI_COMM_WORLD

  • Default communicator group:

    The Communicator Group MPI_COMM_WORLD consists of ALL MPI processes that the program is using This Communicator Group MPI_COMM_WORLD is known as the default MPI communication group.

  ---

  ---

• Most commonly used MPI functions
  • Here are a number of very frequently used MPI functions

    Function Name	Usage
    MPI_Init(int *argc, char*argv[])	Initialize the MPI system - must be the first statement in an MPI program
    MPI_Finalize()	Terminates the MPI system
    MPI_Comm_size(MPI_Comm comm, int *size)	Determines the size of the group associated with the communictor comm The size is returned in the integer variable size
    MPI_Comm_rank(MPI_Comm comm, int *rank)	Determines the rank of the calling process in the communicator comm The rank is returned in the integer variable rank
    MPI_Send(void *buff,                    int count,                    MPI_Datatype type,                    int dest,                    int tag,                    int comm)	Send a point-to-point message to process dest in the communication group comm The message is stored at memory location buff and consists of count items of datatype type The message will be tagged with the tag value tag The MPI_Send() function will only return if the message sent has been received by the destination. (It's safe to reuse the buffer buff right away).
    MPI_Recv(void *buff,                    int count,                    MPI_Datatype type,                    int source,                    int tag,                    int comm,                    MPI_Status *status)	Receive a point-to-point message The message MUST BE from the process source in the communication group comm AND the message MUST BE tagged with the tag value tag The message received will be stored at memory location buff which will have space to store count items of datatype type When the function exits, the exit status is stored in status. Information about the received message is returned in a status variable, e.g.,: The received message tag is status.MPI_TAG and The rank of the sending process (of the message) is status.MPI_SOURCE. In most cases, you know the structure of data received and then you can ignore the status value... If you pass NULL as status parameter, MPI will not return the status value. The MPI_Recv() function will only return if the desired message (from source with tag tag) has been received - or exits with an error code...

  ---

  ---

  ---

  ---

• Hello World in MPI: version 2
  • OK, we are ready for a real MPI program: hello world....

    Well, printing "hello world" is not very illustrative in MPI

    We will make processes send "hello" message to each other...

  • The following MPI Program will do the following:

    The process 0 is the master process 0 will send a "Hello X" message to the process X (X = 1, 2, ...) Process X will reply back to process 0 with "Hello 0, Process is X present and accounted for". Process 0 will print out Process X's reply

    ---

  • Example Program:

    #include "mpi.h" #include <stdio.h> #include <string.h> int main(int argc, char **argv) { char idstr[32]; char buff[128]; int numprocs; int myid; int i; MPI_Status stat; /********************************************* Begin program *********************************************/ MPI_Init(&argc,&argv); // Initialize MPI_Comm_size(MPI_COMM_WORLD, &numprocs); // Get # processors MPI_Comm_rank(MPI_COMM_WORLD, &myid); // Get my rank (id) if( myid == 0 ) { // Master printf("WE have %d processors\n", numprocs); for( i = 1; i < numprocs; i++) { sprintf(buff, "Hello %d", i); MPI_Send(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD); } for( i = 1; i < numprocs; i++) { MPI_Recv(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD, &stat); printf("%s\n", buff); } } else { // Slave MPI_Recv(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &stat); sprintf(idstr, "Hello 0, Processor %d is present and accounted for !", myid); MPI_Send(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD); } MPI_Finalize(); }

    ---

  • Example Program: (Demo above code)                        
    • Prog file: click here

    To compile the program:

    ssh lab1a cd sunhome/MPI mpicc -o Hello2 Hello2.c

    To run the program:

    mpirun -np 4 ./Hello2 Don't forget the period (.) before the Hello ! You MUST give absolute paths to "mpirun"

  ---

  ---

  ---

  ---

  ---

  ---

• Behind the scenes: executing an MPI program
  • Here is how the user is connect to an MPI system:

    Each computer in the MPI cluster has a unique identofier

    The user interacts with the MPI console process (always runs on MPI node 0) that reads user's command and execute them.

    ---

  • When you run a program using mpirun, the program file is loaded from disk into all nodes in the MPI system:

    ---

  • MPI nodes (a computer in the MPI cluster) can send each other messages.

    E.g.: how node 0 send message to node 2:

    Note:

    The message is sent over the interconnection network The message contains the sender ID and receiver ID Only the intended receiver will accept the message !!!

  ---

  ---

• Summary of MPI
  • When you invoke:

    mpirun -np 4 ./Hello

    The SAME program (./Hello) will be run on 4 different MPI processors (because of the option x-np 4)

  • Each MPI process on the different processors will be assigned with a unique ID: 0, 1, 2, 3

  • Only process 0 will be able to use print functions to output to the terminal - the other processes must not try to print. (The behavior will be system dependent).

  • The processes can exhibit different computational behavior by using its rank.

  • MPI provides a communication facility for processes to exchange data with pne another.

  • So the next few webpages will basically discuss MPI's communication facility....

  ---

  ---

• External Resources on MPI
  • Short MPI Tutorial page: click here
  • Short MPI Tutorial page: click here
  • LONG MPI Tutorial page: click here