Glossary
Transformers: Deep Neural Network Architectures for Natural Language Processing

Refinement; fine-tuning

Transfer learning. A model trained for a specific task (e.g., a language model task) can be transferred to another task by extending the training with different types of data or annotations.

Annotation; labeling

Metadata that may pertain to a text, a sentence, or even individual words. It may be linguistic in nature (morphological, syntactic, semantic) or represent the output of a processing task (for example, the polarity of a text or the semantic equivalence between two sentences). Annotated data is necessary to guide supervised learning.

Transfer learning

Training a probabilistic or neural model using annotated texts for a specific task, domain, or language A, followed by its application to process texts from another domain or language B.

Learning with a few examples; few-shot learning

Learning with minimal training using...