Overview
Direct Answer
A neural network architecture comprising an encoder–decoder framework that transforms an input sequence into an output sequence of different length and structure. Originally developed for machine translation, this design pattern has become foundational for sequence-to-sequence transformation tasks across natural language processing.
How It Works
The encoder processes an input sequence token-by-token, compressing information into a fixed-size context vector or attention mechanism. The decoder then consumes this representation to generate the output sequence autoregressively, predicting one token at a time based on the encoder state and previously generated tokens. Attention mechanisms allow the decoder to selectively focus on relevant input positions during generation, significantly improving translation quality and handling of long sequences.
Why It Matters
Organisations deploy this architecture to automate labour-intensive language tasks with high accuracy, reducing operational costs and latency in customer-facing systems. The pattern's flexibility enables handling variable-length inputs and outputs, which is essential for real-world applications where sentence structure and length differ between source and target languages.
Common Applications
Primary use cases include machine translation services, automated summarisation of documents and customer feedback, dialogue systems, and code generation. Question-answering systems and image captioning also leverage this architecture by adapting the encoder to process alternative modalities.
Key Considerations
The fixed context vector can become a bottleneck for very long sequences, though attention mechanisms mitigate this problem. Computational cost during inference—particularly beam search decoding—and exposure bias during training remain important tradeoffs requiring careful tuning and regularisation strategies.
Cross-References(1)
More in Natural Language Processing
Intent Detection
Generation & TranslationThe classification of user utterances into predefined categories representing the user's goal or purpose, a fundamental component of conversational AI and chatbot systems.
Conversational AI
Generation & TranslationAI systems designed to engage in natural, context-aware dialogue with humans across multiple turns.
Hallucination Detection
Semantics & RepresentationTechniques for identifying when AI language models generate plausible but factually incorrect or unsupported content.
Machine Translation
Generation & TranslationThe use of AI to automatically translate text or speech from one natural language to another.
Dialogue System
Generation & TranslationA computer system designed to converse with humans, encompassing task-oriented and open-domain conversation.
Instruction Tuning
Semantics & RepresentationTraining a language model to follow natural language instructions by fine-tuning on instruction-response pairs.
Aspect-Based Sentiment Analysis
Text AnalysisA fine-grained sentiment analysis approach that identifies opinions directed at specific aspects or features of an entity, such as a product's price, quality, or design.
Large Language Model
Semantics & RepresentationA neural network trained on massive text corpora that can generate, understand, and reason about natural language.