Weakly supervised semantic segmentation based on EM algorithm with localization clues

Deep convolutional neural networks have achieved excellent performance in image semantic segmentation with strong pixel-level annotations. However, pixel-level annotations are very expensive and time-consuming. To overcome this problem, we propose a localization clues guided Expectation-Maximization (LCEM) method to optimize segmentation network parameters with image-level labels. Localization clues provide useful cues to infer pixel labels and guide the Expectation-Maximization (EM) algorithm to more accurate network parameters. The proposed LCEM method consists of three steps: (i) Initialization, (ii) latent posterior estimation with the aid of object localization clues (E step), and (iii) update the network parameters using a new object function that incorporates object clues (M step). We also develop a hybrid training strategy to learn the network parameters. Extensive experimental results validate that the proposed method outperforms current state-of-the-art approaches on the challenging PASCAL VOC 2012 image segmentation benchmark for weakly supervised object segmentation.