Class-Agnostic Detection of Unknown Objects from Foreground Improves Robust Open World Object Detection

Open world object detection (OWOD) is a challenging task that requires object detectors to not only detect known object categories but also identify unknown objects. Most OWOD methods adopt pseudo-labeling strategies to discriminate unknown objects in the training set. However, the noisy pseudo labels of unknown objects harm the performance of the model on the known categories. To mitigate the negative effects of inaccurate pseudo labels, we propose an OWOD framework comprising a class-specific detector (CSD) and a class-agnostic detector (CAD). The CAD detects the foreground objects, and we consider a foreground object to be unknown if it does not overlap significantly with any known object discovered by the CSD. We supervise the training of CSD using only the reliable labels of the known category and thus maintain a high localization quality of the known categories. To better discover the foreground objects, we propose to enhance the performance of CAD by incorporating semantic segmentation and prompt-based image segmentation. Our approach demonstrates SOTA performance on M-OWODB and S-OWODB.