---

Introduction to Interconnection Networks

---

• MIMD Parallel Computers
  • A MIMD computer fetches multiple instructions at the same time and execute multiple instructions at the same time

    ---

    ---

  • In order to fetches multiple instructions at the same time, we need a

    Complex communication network

    between the CPU and the memory

  ---

  ---

  ---

  ---

• Interconnecting CPU to memory
  • There are 2 common MIMD architectures:
    1. Shared Memory MIMD:

       NOTE: The memory is split into multiple banks to allow simultaneous transfers of values

       Values stored in different memory banks can be transfered from/to CPUs simultaneously

       ---

       Strength/Weakness of the shared memory approach:

       The interconnection network is extremely fast But at the same time, the interconnection network is very expensive to make

       ---

       ---

       ---

       ---

    2. Message Passing MIMD:

       This architecture is commonly known as networked computers

       ---

       Strength/Weakness of the Message passing approach:

       The interconnection network is slower But at the same time, the interconnection network is cheaper to make It's so cheap that you can build one at home using a Ethernet switch All you need is to download and install the MPI library

       ---

       Notice that in both architectures, the "components" are interconnected by a interconnection network

  ---

  ---

• Differences between Shared-Memory MIMD and Message-Passing MIMD
  • Notice that:
    • The inter-connection networks of the shared-memory MIMD computer connects a CPU to a memory module or a memory module to a CPU

      This kind of network is very fast --- over 100,000,000,000 (100 Gbps) bits per second

    • The inter-connection networks of the message-passing MIMD computer connects two different (SISD or "normal") computers

      This kind of network can be found on store shelft It's the kind of network that is used to connect different computers with each other. The data rate is is low --- from 100,000,000 (100 Mbps) bits per second to 10,000,000,000 (10 Gbps) bits per second.

    ---

    ---

  • Another difference between these 2 types of networks is:
    • The data transfered in shared-memory MIMD computer is:
      • a memory address from a CPU to a memory module or
      • a word (stored in memory) from a memory module to a CPU

      The amount of data transfer is fixed size

    • The data transfered in Message-passing MIMD computer is:
      • a message from a computer to another computer

      The amount of data transfer is variable size

    ---

  • The fixed size messages in shared-memory MIMD and variable size messages in message-passing MIMD lead to different ways to solutions of inter-connection networks to these computers.