3. Deep neural networks

3.1 Features

Deep neural networks have a large number of hidden layers. There are several types of deep neural network:

• DNNs are typically unidirectional, with calculations propagating from input to output. The learning phase is also carried out from inputs to outputs, adjusting the weights of the various layers step by step, from the last to the first;

• in recurrent neural networks (RNN), information can propagate in both directions;

• convolutional neural networks (CNNs) include convolutions. They are notably used in computer vision. Residual neural networks are a special case of CNNs, where connections enable hidden layers to be skipped.

...