Deep learning for 3D images

Deep learning is a discipline that uses artificial neural networks to automatically learn transformations. Over the past decade, this discipline has revolutionized various fields of data science, such as image analysis and natural language processing, to the point of sparking a revival in artificial intelligence (AI). This technological revolution has prompted major companies to recruit researchers and engineers at a premium to build up or strengthen their AI teams. Numerous start-ups have also been created to develop solutions to problems which, just a few years ago, were considered out of reach.

Three-dimensional (3D) images are also playing an increasingly important role in industrial applications, thanks to advances in increasingly high-performance acquisition methods such as X-ray computed tomography, magnetic resonance imaging and laser remote sensing (more commonly known as LiDAR, from Light Detection And Ranging). Another area of research concerns the extraction of 3D information from 2D images.

It is therefore only natural that applications of deep learning to 3D images have been developed in recent years. The aim of this article is to present these methods in a synthetic and accessible way. To this end, we begin by introducing the different 3D representations considered: here, we restrict ourselves to representations in the form of three-dimensional arrays or graphs. We then briefly present the basics of deep learning for images and graphs, and explain how they are applied to 3D images. In the next section, we turn to a more forward-looking topic: 3D image generation. Finally, before concluding, we discuss the prospects and challenges of these methods.