Introduction to parallelism and parallel architectures

The notion of parallelism - using several processors or hardware operators to run one or more programs - is an old one. Multiprocessors date back to the 1960s. From then until the end of the 1990s, parallel architectures were used for applications requiring computing power that single-processor systems were unable to provide. These included mainframes and servers on the one hand, and vector and then parallel machines used for high-performance scientific computing on the other. The 1980s saw the emergence of a number of companies offering parallel machines, which soon disappeared. The main reason for this was the exponential growth in microprocessor performance, used in PCs and multiprocessor servers. The massive use of parallelism was limited to very large-scale numerical simulation applications with massively parallel architectures. The early 2000s, with the limitations of single-processors and the "wall of heat", completely changed the situation (see [H 1 058] ). In 2016, multi-core processors can be found in hardware architectures for all types of components: mobile devices (smartphones, tablets), embedded systems, televisions, laptops and desktop PCs, right up to parallel machines and supercomputers for very high performance.

In this article, we introduce the notion of parallelism, present the different types of parallelism and the different forms of parallel architectures. While programming parallel machines has long been reserved for specialists, any programmer should now master the essential notions of parallel programming to take advantage of the possibilities offered by these architectures. We present the parallel extensions to commonly used programming languages, and the programming models developed to bring parallel programming "closer" to sequential programming techniques, while taking into account the specific features of parallel architectures. Finally, parallel architectures are particularly interesting in terms of performance. To optimize these performances and/or reduce energy consumption, it is necessary to model both the parallelism existing in an application and the parallel architectures. We therefore examine the metrics used to evaluate or predict performance, and the main laws that govern them.