---

Introduction to Parallel Processing

---

• What is Parallel Computing ?
  • Parallel computing is NOT running multiple programs at the same time

    If parallel computing were defined as "running multiple programs at the same time" we can do that by running multiple programs on multiple computers and be done with it...)

  • Parallel computing is NOT running the SAME program simultaneously on multiple computers

    If you run the SAME program simultaneously on multiple computers, you will get the SAME output multiple times - it's not parallel programming, it's called wasting CPU cyles

  • Parallel programming is harnassing the computing from multiple CPUs to achieve a common computational goal

    It involves dividing up the work among multiple CPUs and possibly combining the results of the multiple computations to produce the final outcome

  ---

  ---

• Application Programming Interfaces for Parallel Computing
  • It used to be that every parallel computing system has its own Application Programming Interface.... (used to be a jungle :-))

  • Over time, two main stream APIs have emerged:
    • Posix Threads (PThreads) for shared memory multi-processors

      (There are other API's the are similar to PThreads, such as OMP)

    • Message Passing Interface for message passing multi-processors

      (There are other API's the are similar to MPI, such as Parallel Virtual Machine (PVM))

    ---

    ---

  • We will now study parallel programming using Posix Threads (PThreads)

    If time allows, we will also study MPI