Information carried by a signal

The future evolution of a random natural signal cannot be predicted, so knowledge of its value at the present moment provides information: the signal is said to carry information. This quantity, which we perceive intuitively, is not easy to define quantitatively, but it is natural to attribute certain properties to it:

• is a positive quantity;

• it is additive: every new signal brings a quantity of information that is added to that which we already possess;

• it is linked to the probability associated with the event: for example, learning the winning number of the National Lottery (one million numbers, probability of one of them: 10 ^–6 ) requires a greater amount of information than learning the result of a coin toss (probability 1/2...