7. Deep neural networks

Around 2006, acoustic recognition models were significantly improved thanks to Deep Neural Networks (DNNs), with a far greater number of hidden layers than traditional multi-layer perceptrons (several dozen or more, with thousands of nodes in the hidden layers). These networks, inspired by the functioning of the animal cortex, are capable of learning much more complex functions than ever before. A possible learning algorithm for these deep networks, proposed by G. Hinton, is of the semi-supervised type. The principle is to initialize the weights of each layer's connections in an unsupervised way, then to adapt the whole network in a supervised way. DNNs have proven their worth in a wide variety of fields, including speech recognition, text processing, computer vision and diagnostics. We should also mention the precursors of these deep models designed for image processing and handwriting...