The GPU programming environment

This is what a computer that contains a graphics card computer look like:

 

There are actually 2 computer systems !!!

The GPU programming environment

There is a traditional computer system consisting of the CPU + RAM:

 

You can see this computer system on the left side

The GPU programming environment

There is a GPU computer system consisting of the GPU + "device memory":

 

You can see this computer system in the center (it is sticking out of the mother board)

Characteristics of the GPU (computer system)
 

  • GPU is relatively slower compared to a CPU

  • However:

    • a GPU can perform 10000's of (the same) compuations (at a slower pace) at the same time !!!

  • Therefore:

      • The GPU will outperform the CPU if the program performs a lot of same computations on different values

    Example: 

        for (int i = 0; i < 10000000; i++ )             
     C[i] = A[i] + B[i];  // Many add operations 
                          // on different values

Original usage for the GPU: The image rendering problem
 

  • Image Rendering is the process of generating a photo-realistic image using a 2D or 3D model by means of a computer algorithm.

  • An image consists of many (millions) of points (called pixels):

    • Each pixel has 3 values:

      • The red intensity
      • The green intensity
      • The blue intensity

  • Image rendering algorithms perform the same (series of) operations on to find these values for each pixel (but using different inputs due to the position of the pixel in the image)

  • The SIMD model is highly suitable to speed up image rendering algorithms

Physical view of a computer with a GPU subsystem

Physical appearance of a GPU:

 

The GPU and device memory are inside the casing

Logical view of a computer with a GPU subsystem

Logical structure of a GPU:

 

Left: host ("ordinary" CPU) computer    --- Right: the GPU computer

Note: a DMA processor is used to the transfer data between host memory and device memory

Architecture view of a computer with a GPU subsystem

Architectural differences between the host (CPU) computer and the GPU computer:

 
 

(1) The CPU fetches 1 instruction (from RAM) and executes the instruction on 1 (pair) of input data

Architecture view of a computer with a GPU subsystem

Architectural differences between the host (CPU) computer and the GPU computer:

 
 

(2) The GPU fetches 1 instruction (from Device Memory) and executes the instruction on many (pair) of input data

Architecture view of a computer with a GPU subsystem

Data transfer between the host (CPU) computer and the GPU computer:

 
 

(3) The CPU controls the data transfer operations between RAM and device memory

Architecture view of a computer with a GPU subsystem

How users interact with a GPU capable computer:

 
 

The user interacts with keyboard and terminal attached to the host computer

Architecture view of a computer with a GPU subsystem

How users interact with a GPU capable computer:

 
 

The user controls the GPU computer through an Application Programming Interface (API) (e.g.: CUDA library functions)