Learning without forgetting in NLP: Deep learning based transformer models for lifelong task incremental learning

This study investigates task incremental learning for developing transformer-based natural language processing (NLP) models capable of sequentially learning new tasks without forgetting previously acquired knowledge. It focuses on enabling models to learn from abusive comment detection to offensive comment detection while maintaining previous knowledge. Pre-trained transformer models such as ERT, ALBERT, RoBERTa, and DistilBERT were integrated with continual learning strategies, including rehearsal, pseudo-rehearsal, and regularization-based methods. Their effectiveness was evaluated on sequential abusive and offensive comment detection tasks using measures of classification accuracy and knowledge retention across multiple datasets. Among the evaluated models, BERT integrated with the Learning Without Forgetting (LwF) approach achieved the best trade-off between stability and plasticity, attaining accuracies of 91.06% for abusive and 98.8% for offensive comment detection. This demonstrates the ability of continual learning to avoid catastrophic forgetting in sequential NLP tasks. Task incremental learning enables transformer models to adapt to new linguistic challenges while retaining prior knowledge, supporting lifelong learning. The proposed framework provides valuable insights for building adaptable NLP systems applicable to content moderation, toxic speech detection, and other evolving language-based applications.