SMOTE — Technology Wiki

Overview

Direct Answer

SMOTE is a data preprocessing technique that addresses class imbalance by generating synthetic training examples in the feature space of the minority class, rather than simply duplicating existing minority instances. It uses k-nearest neighbours to create new synthetic samples along the line segments connecting minority class examples.

How It Works

The algorithm identifies minority class samples and, for each one, locates its k-nearest neighbours (typically k=5) within the same class. New synthetic samples are then generated by randomly interpolating between a minority instance and one of its selected neighbours, positioning them at random points along the connecting line in feature space. This process is repeated until the desired balance ratio is achieved.

Why It Matters

Class imbalance severely degrades classifier performance on minority classes, leading to poor recall and F1-scores in critical domains such as fraud detection, disease diagnosis, and anomaly identification. By synthesising rather than replicating examples, the technique increases effective training set size whilst enabling classifiers to learn decision boundaries more effectively without overfitting to genuine minority patterns.

Common Applications

Applications include credit card fraud detection, medical diagnosis with rare diseases, network intrusion detection, and manufacturing defect identification. Telecommunications and banking sectors regularly employ the technique to improve detection of rare but costly adverse events.

Key Considerations

The method assumes minority class samples are sufficiently dense to form meaningful neighbourhoods; sparse or highly scattered minority data may produce poor-quality synthetics. Generated samples exist in interpolated regions that may not reflect true underlying data distribution, and parameter tuning (particularly k and over-sampling ratio) significantly influences results.

Cross-References(1)

Machine Learning

Class Imbalance

Related in Feature Engineering & Selection

Lasso Regression

A regularised regression technique that adds an L1 penalty, enabling feature selection by driving some coefficients to zero.

Feature Engineering

The process of using domain knowledge to create, select, and transform input variables to improve model performance.

Data Augmentation

Techniques that artificially increase the size and diversity of training data through transformations like rotation, flipping, and cropping.

Class Imbalance

A situation where the distribution of classes in a dataset is significantly skewed, with some classes vastly outnumbering others.

Label Noise

Errors or inconsistencies in the annotations of training data that can degrade model performance and lead to unreliable predictions if not properly addressed.

More in Machine Learning

Semi-Supervised Learning

Advanced Methods

A learning approach that combines a small amount of labelled data with a large amount of unlabelled data during training.

Cross-Validation

Training Techniques

A resampling technique that partitions data into subsets, training on some and validating on others to assess model generalisation.

Overfitting

Training Techniques

When a model learns the training data too well, including noise, resulting in poor performance on unseen data.

Transfer Learning

Advanced Methods

A technique where knowledge gained from training on one task is applied to a different but related task.

Support Vector Machine

Supervised Learning

A supervised learning algorithm that finds the optimal hyperplane to separate different classes in high-dimensional space.

Hierarchical Clustering

Unsupervised Learning

A clustering method that builds a tree-like hierarchy of clusters through successive merging or splitting of groups.

K-Means Clustering

Unsupervised Learning

A partitioning algorithm that divides data into k clusters by minimising the distance between points and their cluster centroids.

Ensemble Methods

MLOps & Production

Machine learning techniques that combine multiple models to produce better predictive performance than any single model, including bagging, boosting, and stacking approaches.