1. General framework for statistical machine learning

1.1 Artificial learning = statistics + optimization

Statistical artificial learning is a subset of Artificial Intelligence, and corresponds to various methods that rely on empirical modeling (i.e., based on examples or data) to perform automated classification or continuous approximation (regression), or partitioning. What sets it apart from "knowledge" models (obtained by putting things into equations, as in physics, for example) is that it is "data-driven".

In this sense, most engineers have already learned statistics "without knowing it". One of the simplest and oldest examples is indeed linear regression, which involves approximating a relationship between a quantity and a set of "explanatory variables" (or "inputs") using the "least squares" technique:...