Neural Network — Technology Wiki

Overview

Direct Answer

A neural network is a computational model comprised of interconnected nodes (neurons) organised in layers that learn to recognise patterns through iterative adjustment of connection weights. It mimics the signal-processing behaviour of biological brains to approximate complex functions from training data.

How It Works

Neurons receive weighted inputs, apply an activation function, and forward outputs to subsequent layers in a process called forward propagation. During training, backpropagation calculates gradients of a loss function with respect to each weight, enabling optimisation algorithms to iteratively minimise prediction error. This layered architecture allows the model to learn hierarchical feature representations automatically.

Why It Matters

Neural networks achieve superior accuracy on unstructured data—images, text, audio—compared to traditional machine learning, directly reducing development time and operational costs for organisations. Their ability to discover non-linear relationships without explicit feature engineering accelerates deployment of predictive systems in competitive industries.

Common Applications

Applications span computer vision (image classification, object detection), natural language processing (machine translation, sentiment analysis), speech recognition, and recommendation systems in finance, healthcare, e-commerce, and telecommunications sectors.

Key Considerations

Training requires substantial computational resources and labelled data; interpretability remains limited in deep architectures, complicating regulatory compliance and debugging. Practitioners must carefully manage overfitting risk and validate performance across diverse datasets to ensure generalisation.

Referenced By24 terms mention Neural Network

Other entries in the wiki whose definition references Neural Network — useful for understanding how this concept connects across Deep Learning and adjacent domains.

Activation Function·Deep Learning Attention Mechanism·Deep Learning Autoencoder·Deep Learning Backpropagation·Machine Learning Batch Normalisation·Deep Learning Capsule Network·Deep Learning Convolutional Layer·Deep Learning Encoder-Decoder Architecture·Deep Learning Fully Connected Layer·Deep Learning Graph Neural Network·Deep Learning Large Language Model·Natural Language Processing Long Short-Term Memory·Deep Learning Model Pruning·Artificial Intelligence Neural Architecture Search·Artificial Intelligence Neural Processing Unit·Artificial Intelligence Pipeline Parallelism·Deep Learning Pooling Layer·Deep Learning Quantisation·Artificial Intelligence Quantum Neural Network·Quantum Computing Recurrent Neural Network·Deep Learning Residual Network·Deep Learning Seq2Seq Model·Natural Language Processing Skip Connection·Deep Learning Text Embedding Model·Natural Language Processing

Related in Architectures

Deep Learning

A subset of machine learning using neural networks with multiple layers to learn hierarchical representations of data.

Convolutional Neural Network

A deep learning architecture designed for processing structured grid data like images, using convolutional filters to detect features.

Recurrent Neural Network

A neural network architecture where connections between nodes form directed cycles, enabling processing of sequential data.

Long Short-Term Memory

A recurrent neural network architecture designed to learn long-term dependencies by using gating mechanisms to control information flow.

Gated Recurrent Unit

A simplified variant of LSTM that combines the forget and input gates into a single update gate.

Transformer

A neural network architecture based entirely on attention mechanisms, eliminating recurrence and enabling parallel processing of sequences.

Attention Mechanism

A neural network component that learns to focus on relevant parts of the input when producing each element of the output.

Encoder-Decoder Architecture

A neural network design where an encoder processes input into a fixed representation and a decoder generates output from it.

Autoencoder

A neural network trained to encode input data into a compressed representation and then decode it back to reconstruct the original.

Variational Autoencoder

A generative model that learns a probabilistic latent space representation, enabling generation of new data samples.

Batch Normalisation

A technique that normalises layer inputs during training to stabilise and accelerate deep neural network learning.

Embedding

A learned dense vector representation of discrete data (like words or categories) in a continuous vector space.

More in Deep Learning

LoRA

Language Models

Low-Rank Adaptation — a parameter-efficient fine-tuning technique that adds trainable low-rank matrices to frozen pretrained weights.

Fine-Tuning

Language Models

The process of adapting a pre-trained model to a specific task by continuing training on a smaller task-specific dataset, transferring learned representations to new domains.

Rotary Positional Encoding

Training & Optimisation

A position encoding method that encodes absolute position with a rotation matrix and naturally incorporates relative position information into attention computations.

Diffusion Model

Generative Models

A generative model that learns to reverse a gradual noising process, generating high-quality samples from random noise.

Dropout

Training & Optimisation

A regularisation technique that randomly deactivates neurons during training to prevent co-adaptation and reduce overfitting.

Vanishing Gradient

Architectures

A problem in deep networks where gradients become extremely small during backpropagation, preventing earlier layers from learning.

Pre-Training

Language Models

The initial phase of training a deep learning model on a large unlabelled corpus using self-supervised objectives, establishing general-purpose representations for downstream adaptation.

Residual Connection

Training & Optimisation

A skip connection that adds a layer's input directly to its output, enabling gradient flow through deep networks and allowing training of architectures with hundreds of layers.