Taxonomy-Guided Group Prototype Clustering for Open-Set Plankton Recognition From Imbalanced Underwater Imagery

Automated in situ plankton recognition is crucial for marine ecological monitoring but faces the compounded challenges of open-set recognition, where most captured particles are non-plankton, and long-tailed, imbalanced data distributions. Existing methods often rely on external auxiliary out-ofdistribution (OOD) datasets for training or treat classification and OOD detection as separate tasks, limiting their practicality. To address these limitations, we propose a novel taxonomy-guided Grouped Supervised Prototype Clustering (GSPC) framework. Our method first introduces a domain knowledge-informed pregrouping mechanism, clustering 78 known plankton classes into 9 biologically coherent groups based on taxonomic principles. Building upon this structure, we develop a dual-branch network optimized by a unified loss function. This function combines a group-based cross-entropy loss with novel class-and grouplevel prototype contrastive losses, jointly enforcing intra-class compactness and inter-group separation in the feature space to create a discriminative representation robust to unknown samples. Extensive experiments on the DYB-PlanktonNet dataset demonstrate that our framework simultaneously advances both closed-set classification and open-set detection. It achieves a state-of-the-art FPR95 of 8.34% and an AUROC of 98.30% without requiring any external auxiliary OOD data for training. This work provides a robust, self-sufficient, and biologically interpretable solution for automated plankton monitoring in complex marine environments.