1. Discrete equations associated with Markov chains in discrete time and space

This first paragraph, in the simple framework of Markov chains with a discrete state space, allows us to highlight the link between this type of stochastic process and certain problems that appear as discrete versions of boundary, evolutionary or stationary problems. These problems can be solved iteratively, and the probabilistic point of view consists in expressing the solution of the problems considered in an explicit form using the expectation (or conditional expectation) of the associated Markov chain. This simple framework can serve as a guide to what will be done in the following paragraphs for continuous problems. The results of this paragraph can be found in the first chapter of