5. Glossary

Annotation; Labeling

Annotated data is needed to guide supervised learning. Annotations can relate to a text, a sentence, or even isolated words; they can be linguistic in nature (morphological, syntactic, semantic), or represent the output of a processing task (e.g. the polarity of a text, or the semantic equivalence between two sentences).

Refining; Finetuning

A model trained for a particular task (e.g. one on a language model task) can be transferred to another task by extending the training with other types of data or annotations: this is the refinement stage. In this way, the parameters of a model like BERT can be specialized using a few examples from a sentiment analysis task. Model refinement is one method of transfer learning.

Transfer Learning

Transfer learning...