MIMD Parallel Computers

  • A MIMD (Multiple Instruction Multiple Datastream) computer can:

      • execute multiple instructions and
      • each instruction can operate on different input data

    Basically:

      • A MIMD computer consists of multiple independent CPUs

The 2 architectures of MIMD computers

There are 2 common MIMD computer architectures:

  1. Shared memory MIMD computers:

      • The computer memory (RAM) in a shared memory MIMD computer are accessible by all CPUs in the MIMD computer

        I.e.: each CPU in the MIMD computer will "see" the same memory content

      • The shared memory is the mean (= method) for programs to communicate with each other

  2. Message Passing MIMD computers

      • Each CPU has its own private memory

      • The mean of communication between programs is through (computer) messages

Architecture of the shared memory MIMD computer

The shared memory MIMD computer has a very high speed interconnection network between the CPUs and the memory banks:

The memory consists of multiple memory banks that operate independently to provide higher data transmission rates

Purpose of the interconnection network

Each processor (CPU) need to fetch its own instruction and the operands from memory:

 
 

Each memory module can provide 1 word (32 bits) data to a (any) processor if there is a connection

Purpose of the interconnection network

Processors (CPUs) accessing different memory modules can/may obtain their data simultaneously:

 
 

 

Purpose of the interconnection network

Processors (CPUs) accessing the same memory modules will cause a memory conflict:

 
 

Only one of the access request can be granted !!

Strength/weakness of the shared memory MIMD computer

The interconnection network between the CPUs and the memory banks is extremely fast (tera bits/sec) but is also very costly to manufacture:

Today's super-computers are always shared memory MIMD computers

The GPU also has an interconnection network between the multiprocessors and the device memory

The speed up data transfer, the GPU device memory is partitioned into multiple banks

An interconnection network transfer data between the multiprocessors and the device memory

Architecture of the message passing MIMD computer

The message passing MIMD computer consists of multiple SISD computers interconnected by a (local) computer network (e.g., Ethernet):

 

 

The (local) computer network has lower transmission speed rate compared to the CPU-memory connection network in the shared memory MIMD computer

Strength/weakness of the message passing MIMD computer

The message passing MIMD computer is relatively inexpensive to manufacture but its performance is also lower:

 

 

The message passing MIMD computer is a.k.a. computer cluster ( Wikipedia)