Introduction to Interconnection Networks

MIMD Parallel Computers

A MIMD computer fetches multiple instructions at once and execute multiple instructions at once

There are 2 common MIMD architectures:

Shared Memory MIMD:

Notes:

All CPUs share the same memory
Memory is split into multiple banks to allow simultaneous transfers of data
Data stored in different memory banks may be transfered from/to CPUs simultaneously

Differences between Shared-Memory MIMD and Message-Passing MIMD

Important Difference

The inter-connection networks of the shared-memory MIMD computer connects:

Data moved:

The data transfered in shared-memory MIMD computer is:
- a memory address from a CPU to a memory module or
- a word (stored in memory) from a memory module to a CPU
The amount of data transfer is fixed size

The inter-connection networks of the message-passing MIMD computer connects

Data moved:

The data transfered in Message-passing MIMD computer is:
- a message from a computer to another computer
The amount of data transfer is variable size

The fixed size messages in shared-memory MIMD and variable size messages in message-passing MIMD lead to:

Communication between CPUs in MIMD

The CPU execute program instructions
In order to collaborate in a common task , the different CPUs must exchange data (information)
Shared-memory MIMD machines exchange data by writing and reading a common memory location
Example: CPU 1 sends information by writing to variable x
Example: CPU N receives information by reading from variable x
Message-Passing MIMD machines exchange data by sending and receiving messages over a communication network
Example:
- CPU 1 sends a message over a communication network to CPU N
This form of communication is exactly the same as an "ordinary" computer network (such as the Internet)