Selecting an Optimal Subset of Training Data under Label Noise

Large, labelled datasets used to train machine learning models often contain label noise that can seriously affect the performance of these models. However, it has been shown that a subset of the noisy training data exists on which models can show performance similar to those trained on a noise-free dataset. One of the methods used to identify such subsets is cutstats, which selects candidates by comparing each sample’s features and labels with  those of its neighbourhood. Nevertheless, the performance of cutstats on high-dimensional data has not been fully explored.

A team of researchers from the Representation Learning Lab (RepLab)—Kumar Shubham, Pavan Karjol, Kiran M K, and Prathosh A P—investigated the performance gap between classifiers trained on noisy subsets and those trained on noiseless datasets in high-dimensional settings.

They show that,

  • the performance gap, for cutstats, is determined by the accuracy of the k-NN (k-nearest neighbours) in identifying noiseless samples;
  • incorporating prior knowledge symmetries and associated invariances can preserve the effectiveness of k-NN in high-dimensional settings.

This research work has been accepted for presentation at the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026).

Paper: https://arxiv.org/abs/2605.01874

MENU