Scene-based Graph Convolutional Networks for Federated Multi-Label Classification

Federated multi-label learning can collaboratively train multi-label classification models without compromising user privacy. Compared to multi-class learning, one of the most critical issues of multi-label learning is how to capture the correlations between labels, which is often ignored by existing research on federated multi-label learning. In this paper, a scene-based federated multi-label learning framework is proposed, which effectively utilizes the dependencies among labels for model training on the client-side and aggregates diverse client information on the server-side. Specifically, in the local training phase, a scene recognition module is employed to detect the scene for each image and the corresponding label co-occurrence matrix is used to guide the propagation of image features on the label graph. In the aggregation phase, a scene-aware aggregation method is adopted to enrich the scene-label co-occurrence information of each client. Experiments on PASCAL VOC 2007 and MS-COCO show that our proposed method can significantly improve the accuracy of federated multi-label image classification.