Continual Learning of Medical Image Classification Based on Feature Replay

The wide variety of diseases in clinical diagnosis makes it impractical to develop specific detection algorithms for each disease. Models with continual learning capabilities learn to detect new disease as needed and can eventually detect all diseases learned before. However, there are few researches on continual learning of medical image classification. In this paper, we design two kinds of continual learning tasks of medical image classification and evaluate continual learning methods in the literature. We propose a novel continual learning method based on feature replay. Our method also utilizes multiple conditional generators to improve quality of replayed samples. Comparison with other methods shows that our method achieves higher average accuracy and lower average forgetting. Inception Score and Fréchet Inception Distance show that our method generates better samples which help to overcome catastrophic forgetting significantly.