Dual-Branch Pseudo-Label Refinement for Source-Free Domain Adaptation

Source-free domain adaptation (SFDA) is a practically significant and challenging problem. It aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain with a different data distribution while lacking access to the source domain. The target domain typically comprises two types of data: source-like and domain-specific data. However, identifying source-like data without access to the source domain remains a challenging task. Although some SFDA methods attempt to partition the target domain by using additional classifiers or relying on the prediction confidence of a single model, these approaches often suffer from suboptimal partitioning performance due to the potential biases of a single model towards the target domain. To address the above issue, we propose a novel dual-branch pseudo-label refinement method for SFDA. The proposed method employs a dual-branch framework to identify source-like data and divide the target domain data into two subsets. Following the divide-and-conquer principle, we design specific learning strategies tailored to each subset. We propose using the CLIPN model to identify erroneous samples in the source-like subset and correct them using the vision-language model (VLM). For the domain-specific subset, we introduce a novel prompt-learning technique based on the label space to optimize the label distribution. Both subsets utilize VLMs for pseudo-label refinement at varying degrees, enabling implicit alignment of label distributions. In experiments on three domain adaptation benchmark datasets, our method achieves the latest state-of-the-art results compared to existing approaches, with performance gains of +4.3% and +8.6% on the Office-Home and DomainNet datasets, respectively.