TailAdapt: Continual Learning Under Class Imbalance

Real-world data is rarely balanced. A handful of ‘head’ classes appear more frequently while most ‘tail’ classes appear only a few times. This is hard enough for a model trained once. It is much harder when the model must learn new classes in sequence without forgetting old ones, a setting known as long-tailed class-incremental learning. Frequent classes dominate the model’s updates, and rare classes are the first to be forgotten or never properly learned.

Most existing remedies, such as replaying stored samples or reweighting the loss, correct the classifier but treat the model’s adaptable parameters uniformly. A natural question arises whether this uniform treatment is the real culprit. When prompt tokens are masked in a prompt-tuned vision transformer, head classes continued to show nearly consistent performance even as large fractions of parameters were removed, because they spread information redundantly. Tail classes, by contrast, relied on a small, fragile set of parameters that routine updates tend to overwrite.

To address this, Abhishek Kumar Sinha and Soma Biswas from the IACV Lab, Indian Institute of Science and Nitant Dube from the Space Applications Centre (SAC), Indian Space Research Organisation developed TailAdapt. This is a hierarchical variational framework that groups the learnable parameters (for example, each prompt token or adapter block) and gives each group a latent scale governed by an Inverse Gaussian prior. This scale acts as a learned control on how freely the group can change. Most parameter groups are kept close to what the model has already learned, so earlier knowledge is preserved. Only the few groups that the new data genuinely needs are allowed to change significantly. After each session, the model’s accumulated knowledge is blended in gradually rather than all at once, which stops it from becoming too rigid and keeps it able to learn new classes later.

TailAdapt works as a lightweight add-on to both prompt-based methods and adapter-based methods, without changing the backbone or the replay strategy. Across CIFAR-100, ImageNet-R, and CUB-200, it improved accuracy and reduced forgetting for most baselines, and was competitive with or better than the recent augmentation-based method PANDA. These findings suggest that when learning from imbalanced data over time, it is not enough to focus on the data alone; how freely each part of the model is allowed to adapt also matters.

This work will be presented at the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026).

Paper: https://darkbird1.github.io/assets/pdf/tailadapt.pdf

MENU