8. Programming parallel architectures

Parallel execution of several programs requires them to be generated by a compiler or written by a programmer. Ideally, the programmer should not worry about parallelism and leave it to a compiler to automatically parallelize the program. This is particularly true if you want to run programs written for sequential machines on a parallel machine. However, automatic parallelization comes up against the difficult problem of extracting parallelism. Another, even deeper, problem is that it involves parallelizing programs derived from algorithms for which no effort has been made to exploit parallelism. The algorithms used may even be intrinsically sequential. If these programs have not been designed to be "parallel", it is understandably difficult to achieve a very high degree of parallelism with this approach. This is the reason behind the development and maintenance of parallel programming...