Tokenisation

Large Language Model

A neural network trained on massive text corpora that can generate, understand, and reason about natural language.

GPT

Generative Pre-trained Transformer — a family of autoregressive language models that generate text by predicting the next token.

BERT

Bidirectional Encoder Representations from Transformers — a language model that understands context by reading text in both directions.

Language Model

A probabilistic model that assigns probabilities to sequences of words, enabling prediction of the next word in a sequence.

Contextual Embedding

Word representations that change based on surrounding context, capturing polysemy and contextual meaning.

Word2Vec

A neural network model that learns distributed word representations by predicting surrounding context words.

GloVe

Global Vectors for Word Representation — an unsupervised learning algorithm for obtaining word vector representations from aggregated word co-occurrence statistics.

Instruction Tuning

Training a language model to follow natural language instructions by fine-tuning on instruction-response pairs.

RLHF

Reinforcement Learning from Human Feedback — a technique for aligning language models with human preferences through reward modelling.

Grounding

Connecting language model outputs to real-world knowledge, facts, or data sources to improve factual accuracy.

Hallucination Detection

Techniques for identifying when AI language models generate plausible but factually incorrect or unsupported content.

Prompt Injection

A security vulnerability where malicious inputs manipulate a language model into ignoring its instructions or producing unintended outputs.

The artificial production of human speech from text, also known as text-to-speech.

The computational study of people's opinions, emotions, and attitudes expressed in text.

A text generation strategy that restricts the model to sampling from the K most probable next tokens.

A measure of how closely the meanings of two text passages align, computed through embedding comparison and used in duplicate detection, search, and recommendation systems.

A neural network architecture that maps an input sequence to an output sequence, used in translation and summarisation.

The process of assigning grammatical categories (noun, verb, adjective) to each word in a text.

An unsupervised technique for discovering abstract topics that occur in a collection of documents.

The use of AI to automatically translate text or speech from one natural language to another.