What is parallel computing?

Parallel computing is relative to serial computing. The so-called parallel computing is divided into time parallelism and space parallelism. Time parallelism refers to pipelining technology, and space parallelism refers to multiple processors performing calculations at the same time.

The main research of parallel computing science is spatial parallelism. Spatial parallelism leads to the emergence of two types of parallel machines, which are divided into single instruction stream multiple data streams (SIMD) and multiple instruction stream multiple data streams (MIMD) according to Flynn. Our common serial machine is also called single instruction stream single data stream (SISD).

MIMD machines can be divided into the following five common categories:

Parallel vector processor

Symmetric multiprocessor (SMP)

Massively parallel processor

Workstation cluster (COW)

Distributed Shared Storage Processor (DSM).

[Edit] Memory Access Model

Parallel computers have the following four memory access modes:

Unified memory access model (UMA)

Non-uniform memory access model (NUMA)

Full cache access mode (COMA)

Unified cache non-unified storage access model (CC-NUMA) and non-remote storage access model (NORMA).