2. Termination of asynchronous parallel iterative algorithms

While convergence detection of synchronous parallel iterations does not present any major difficulties, the non-deterministic behavior of asynchronous methods means that in this case there are real difficulties in implementing reliable iteration termination tests. Indeed, the problem of terminating asynchronous parallel iterations is extremely difficult to code, as it involves both applied mathematics and computer science. In mathematical terms, the termination of such methods must occur when the iterated vector is sufficiently close to the solution of the problem. From a computing point of view, we need to take into account the specific nature of multi-processor machine architectures, particularly on distributed systems where communication is by message passing, and in this context the processors only possess local information.

As we shall see in the third article...